EMI PS2 -> Windows

[Gannet]

The PS2 version of EMI supposedly has a number of improvements over the Windows version. What I'd like to do is put these improvements back into the Windows version. I haven't had much success so far but here's what I've learnt...

 

Dialog:

• The PS2 version shows some extra lines of dialog between Guybrush and the catapult operator. There may be extra dialog elsewhere too, though I'm not sure.

• The extra dialog actually does exist in the Windows data but is unavailable in game due to a small error/oversight with regards to the number of lines of dialog displayed at any one time.

• It can be fixed by inserting the "_dialog.lua" script from the PS2 data into the Windows data, but this has a critical side-effect of the game getting stuck whenever you exit a conversation. Properly fixing it would involve editing and recompiling the script.

• The game uses a custom version of lua 3.1 with a modified bytecode. This means that while I can use luadump to decompile and then edit the script, the game won't accept it if I recompile with normal luac 3.1.

 

Models:

• The PS2 version supposedly has "9 times as many polys as the PC version"*.

• The format of the PS2 .meshb files is almost identical to the Windows ones. Once converted they can be put into the Windows data and the game will load them correctly.

• The actual difference observed in-game is negligible. Only a handful of meshes actually have (slightly) higher vertex counts, and none appear any better than the Windows ones.

• The PS2 version appears to render models more smoothly than the Windows version, giving the illusion of more polygons (not something that could be changed in the Windows version).

 

Video:

• The PS2 version has 640x448 mpeg-2 video with 48kHz uncompressed audio, while the Windows version has 640x480 bink video with 22kHz bink audio. The PS2 video suffers from less compression artifacts and the audio is noticeably better.

• With a few tools the PS2 video can be converted to the Windows format with reasonable success. The difference really isn't that great though, especially after recompression.

• If you really want the best quality you're better off just extracting the PS2 video and watching it in a separate player.

 

Sound:

• The PS2 version supposedly has higher quality sound than the Windows version due to "files that weren't as heavily compressed"*.

• Both the PS2 and Windows voices are 16.5kHz mono. The PS2 files are smaller than the Windows ones though, which implies heavier compression.

• Both the PS2 and Windows sound effects are (almost all) 22kHz mono. The PS2 files are compressed though while the Windows ones are not.

• The PS2 music is 24kHz while the Windows music is 44kHz.

• I can find no basis to the claim above. Perhaps she was thinking of the video when she said that.

 

Misc:

• The PS2 version includes a new easter egg mini-game called Monkey Invaders, and keeps track of moves you learn for Monkey Kombat so you can easily review them. This will likely all be script-based with lua so could in theory be added to the Windows version but as with the dialog mentioned above, an appropriate lua compiler would be needed to do this.

• The PS2 version uses a 3D model for the giant monkey head which animates open/closed, instead of being pre-rendered as part of the background image and using videos to open/close. This was because "NTSC does some interesting things to certain colors"*. On a PC I think the pre-rendered stuff is better.

 

Any help or more information regarding any of this would be much appreciated

 

 

* Shara Miller Interview, The SCUMM Bar

[James Isaac]

Nice idea! Here's the specs from that thread, but I don't have the mesh viewer:

 

Show spoiler

(hidden content - requires Javascript to show)

Escape from Monkey Island file specs v0.01
-------------------------------------------
by Benjamin Haisch aka john_doe (john_doe@techie.com)
If you have any questions regarding this text or found out new information just mail me.


Introduction
==============

In this text I try to describe the file formats of the 3d files used in Escape from Monkey Island.
These files are meshes (.meshb), skeletons (.sklb) and animations (.animb).
I haven't yet looked into the costume files (.cosb) which group all the above files together.

Note that I'm a newbie to 3d coding so many terms are probably wrong.


Contents
==========

0.  Common definitons
1.  Binary meshes (.meshb) specs
1.1 Notes about BoneInfoCount
2.  Binary skeleton (.sklb) specs
3.  Binary animation (.animb) specs
3.1 Note on bone operations:
3.2 Note on rotations:



0. Common definitons
======================

vector3d = float x,y,z
vector2d = float x,y

string:
long		length
byte[length]	string data



1. Binary meshes (.meshb) specs
=================================

These files contain the actual 3d model model data, i.e. vertices, normals, faces and texture coordinates and 

optionally information which vertex is connected to which bone in the skeleton.

Start:
--------
string		MeshName		Name of the mesh
vector3d	u[4]			unknown (maybe for collision detection)

long		numtex			Number of textures used
string		texname[numtex]		Texture filenames (incl. path information)

long		u			unknown, is 0x13 in many files
long		numvert			Number of vertices or similar
vector3d	vectors[numvert]	Vertex data
vector3d	normals[numvert]	Normal data

byte		size is numvert * 4	colormap for vertex coloring

vector2d	texcoords[numvert]	texture coords

long		facedatacount		Number of texture groups
				(for faster texturing, switching textures
				only once per texture)

repeated facedatacount times {
long		flags			usually 0x0304
long		textureused		0= not textures; 1= textured
long		texture			Index of texture to be used
				this is only stored if textureused=1
long		facescount		Number of faces
word		faces[facescount]	the faces data
}

long		hasbones		0= mesh has no bones; 1= has bones

if hasbones=0 the file ends here, else there's some more to read:

long		bonecount		Number of bones
string		bonename[bonecount]	Names of the bones
				(they refer to the names specified
				in the skeleton file, .sklb)
long		boneinfocount		Number of "bone infos" (need a better name)
				Stores for each vertex which bone it belongs to.

repeat boneinfocount times {		See description below for more notes
long		incval			(Need better name)
long		bone			The bone the vertex belongs to.
				I don't know yet if this index is into the skeleton
				file or into the bonename array first and from
				there to the correct bone in the skeleton.
float		weight			The weight the assiciated bone has to this vertex.
}

(Note: "Usually" means it's the same in all the files I checked.)


1.1 Notes about BoneInfoCount
-------------------------------

This is where the bone-vertex relations are stored.
Note that a vertex can be associated to more than one bone.
That's why boneinfocount and numvert may differ (but they may be the same, too).

If incval is one, then go to the next vertex. Otherwise if it's 0, then stay at the current vertex.

All weights added must equal to 1.0.


Example (from mk1.meshb):

BoneInfo[0]: Vertex= 0	IncVal=  1   Joint= 130  Weight= 1.000
BoneInfo[1]: Vertex= 1	IncVal=  1   Joint= 130  Weight= 1.000
BoneInfo[2]: Vertex= 2	IncVal=  1   Joint= 130  Weight= 0.871
BoneInfo[3]: Vertex= 2	IncVal=  0   Joint=  91  Weight= 0.129
BoneInfo[4]: Vertex= 3	IncVal=  1   Joint= 130  Weight= 1.000
BoneInfo[5]: Vertex= 4	IncVal=  1   Joint= 130  Weight= 1.000
BoneInfo[6]: Vertex= 5	IncVal=  1   Joint= 130  Weight= 0.749
BoneInfo[7]: Vertex= 5	IncVal=  0   Joint=  91  Weight= 0.251


Here the vertices 0, 1, 3, 4 are each linked to one bone (each with a weight of 1).
The vertices 2 and 5, however, are linked to two bones, each with different weight factors that add up to 1 

(for each vertex).



2. Binary skeleton (.sklb) specs
==================================

These files store the skeleton of the model. Each bone is connected to several vertices of the mesh, so if one 

bone is moved or rotated, so do the attached vertices.

string32 means it's a string with a constant size of 32 bytes, without any length byte.


Start:
--------
long		bonecount		Number of bones in the skeleton

repeat bonecount times {
string32	name			Name of the bone
string32        parent			Name of parent bone
vector3d	position		Initial position of the bone
vector3d	rotation		Initial rotation of the bone
float		angle			Initial rotation angle
}



3. Binary animation (.animb) specs
====================================

These files contain keyframe data. Each bone has its own keyframe chunk. Each chunk can either be a translation 

or a rotation (maybe there are other types as well).


Definitions:
--------------
translation:
vector3d	vec		translation vector
float		time		time this translation is finished

rotation:
quaternion	rot		rotation quaternion
float		time		time this rotation is finished


Start:
--------
string		name		Name of the animation
float		duration	duration of this animation in seconds
long		bones		Number of animated bones (need another name)

repeat bones times {
string			bonename	Name of bone to be animated
				This refers to the bone name stored in the skeleton
long			operation	Operation on this bone (see below)
long			unk1		unknown (often 0x01)
long			unk2		unknown (often 0x00)
long			keyframes	Number of keyframes of this bone

if operation=3
translation[keyframes]	translations	List of translations

if operation=4
rotation[keyframes]	rotations	List of rotations

}


3.1 Note on bone operations:
------------------------------

If operation is 3 this means the bone has to be translated according to the data.
If operation is 4 the bone has to be rotated instead.
Other keyframe types may be possible but I only found these two.


3.2 Note on rotations:
------------------------

I think the rotations are stored as quaternions but they might as well be just an axis as vector3d and an 

rotation angle.

[Benny]

Model viewer:

http://quick.mixnmojo.com/temp/emv-0.5beta.zip

[Gannet]

Thanks guys, great to get that model viewer.

It doesn't open the PS2 mesh files. I took a look at the data and it seems to match the spec except for one differences: the faces data uses longs instead of words (seems everything in the PS2 data must be in multiples of 4). I'll try to convert it and then see if the model viewer and/or the game will load it.

[Gannet]

Cool, it worked. Sadly though, the difference between the models seems to be negligible :/

Where did the idea of "9 times more polygons" come from? I've seen it mentioned on the forums here and on worldofmi.

 

 

Is there any information/tools available for working with the EMI voice files (.wVC inside the bundles)?

[Radogol]

Cool, it worked. Sadly though, the difference between the models seems to be negligible :/

 

How about comparison screenshots, please?

[Gannet]

I haven't looked at all the models but I've only spotted four so far that actually have different vertex counts. Here's Guybrush and Mr. Cheese. In the Windows data, Guybrush has 1036 vertices and Mr. Cheese has 1000. In the PS2 data, Guybrush has 1056 and Mr. Cheese 1156. Guybrush's cuffs and belt are slightly different and his boots are a slightly different colour. Mr. Cheese's hat is rendered slightly differently. That is all.

[Thrik]

Could it be that the PS2 uses a different lighting method to give the illusion of smoother models, or it has some kind of 3D post-processing that smooths some of the vertices? When you look at the hair for example here on the PS2 it looks smoother:

 

http://images.psxextreme.com/screenshots/ps2_monkey_island/ps2_monkey_island_08.jpg

[Radogol]

Windows meshes

PS2 meshes

 

You're right, the difference is minimal.

 

The movies are supposed to be much higher quality on the PS2, although I can't even begin to imagine the size of such a cutscene "patch".

[Gannet]

@Thrik: Yeah, I'd bet that it just renders them more smoothly, which is not something that could be "fixed" in the Windows version. But I'm curious to know whether "nine times" has any basis or if it's just something random someone made up.

 

@Radogol: The voices are supposed to be better too. But without any tools that can even playback the files, I can't even verify if this is true or not let alone convert them across. (I tried Bink player but it only works for a couple of the shorter movies - the main cutscenes won't play).

 

[edit] Okay, I just played the cutscenes in-game and compared to the PS2 video, which plays in VLC. Yes, the PS2 video is higher quality (in terms of compression/artifacts, not model detail) but not hugely. Also, the PS2 video is only 640x448, presumably cut short to display the subtitles below it (in Windows, the subtitles are rendered on top of the video in fullscreen but in a window the video is shorter and they're rendered in the blank space below it). So you're probably losing more than you're gaining anyway.

[James Isaac]

I'm sure I remember reading the "N times as many polygons" thing with official specs about the game too... but can't find anything with google, apart from this:

 

http://webcache.googleusercontent.com/search?q=cache:J8DuPT7mc8QJ:www.mixnmojo.com/features/read.php%3Farticle%3Descapefrommonkeyisland%26page%3D5+%229+times+as+many+polygons%22&cd=3&hl=en&ct=clnk

Re-rendered, allegedly smoother animations - This is something LEC touted endlessly when promoting the PS2 version, stating that there were "9 times as many polygons" in the character models (???) but honestly I've never noticed a difference. The generally blurrier nature of the PS2 version's visuals makes jagged edges around models much less of an issue than with the PC version, however.

[SyntheticGerbil]

This is a great idea, I wouldn't mind a convergence of the best of these two versions at all.

 

Models:

• The PS2 version has been reported by some people to have "nine times more polygons" than the Windows version.

• The format of the PS2 .meshb files is almost identical to the Windows ones. Once converted they can be put into the Windows data and the game will load them correctly.

• The actual difference observed in-game is negligible. Only a handful of meshes actually have (slightly) higher vertex counts, and none appear any 'better' than the Windows ones.

• The PS2 version probably renders models more smoothly than the Windows version, giving the illusion of more polygons. This is not something that could be "fixed" in the Windows version.

 

I'm just curious for anyone who took the time to open the model files, can you list the differences you saw? Sounds like not even all characters or objects are improved.

 

You're right, the difference is minimal.

 

The movies are supposed to be much higher quality on the PS2, although I can't even begin to imagine the size of such a cutscene "patch".

 

Oh man, that's true, but maybe there's some kind of exe that could be make that would require you to have the PS2 disc and just rip them for you and place them over the PC files? I know that's more work, assuming this all happens, but it would be having to go through the agony the Scumm guys used to make you do to convert all the movies from the Feeble Files or download them as they now (sort of grey area) do.

 

@Radogol: The voices are supposed to be better too. But without any tools that can even playback the files, I can't even verify if this is true or not let alone convert them across. (I tried Bink player but it only works for a couple of the shorter movies - the main cutscenes won't play).

 

[edit] Okay, I just played the cutscenes in-game and compared to the PS2 video, which plays in VLC. Yes, the PS2 video is higher quality (in terms of compression/artifacts, not model detail) but not hugely. Also, the PS2 video is only 640x448, presumably cut short to display the subtitles below it (in Windows, the subtitles are rendered on top of the video in fullscreen but in a window the video is shorter and they're rendered in the blank space below it). So you're probably losing more than you're gaining anyway.

 

Hah, funny they didn't take the time to use their "9 times as many polys" models for the PS2 version's movies, but just used the same movie masters without compressing them as much. Can't say I'm surprised.

[Gannet]

@James: Thanks, that's a helpful article. Down the bottom it linked to here, which looks like the official source. It does make it rather confusing though, since I can find no evidence in the data to back up that claim. I'll keep looking though...

 

@SyntheticGerbil: The screenshots posted later show the differences. If/when I find anything more than that, I'll let you know

[SyntheticGerbil]

Ah crap, I completely missed that part. Sorry.

 

Thanks for the screenshots by the way, Gannet. It looks like they may have just chamfered some of the edges I'm guess, but it really doesn't make much of a difference of appearances besides it seems that they did it to get rid of some unnecessary hard shadows (not that I noticed in the first place).

 

Haha, definitely not 9 times more polys.

[LogicDeLuxe]

@James: Thanks, that's a helpful article. Down the bottom it linked to here, which looks like the official source.

That's interesting.

What about the "files that weren't as heavily compressed"? Is there any difference in the music? Has it a higher bitrate, or is this just the videos?

And I'm told that it doesn't have iMuse? Is that true too?

[Gannet]

The music on the Windows version comes in two types: 22kHz or 44kHz mp3s (you choose the quality in the settings). I haven't compared but I wouldn't think the PS2 version would have anything better than that. I believe that comment refers to the video and voice. As for iMuse, I can't determine that from the data but I'm happy believe what others say about it.

 

 

Updates on video:

 

The Windows videos are all in bink format but most have them have a SMUSH header - if I remove this the bink player is happy with it. Also, the PS2 video is not cut off at the bottom as I said earlier but simply squished a little (which is trivial to fix when recompressing with bink). It also has 48kHz uncompressed audio, compared to 22kHz bink audio in the Windows version. It does sound nicer

 

After a struggle I managed to convert the PS2 video to and put it into the Windows version. The result is reasonable though for some reason the audio had to be reduced to 30kHz because the bink compressor didn't like it any higher than that. I'm happy to write up detailed instructions if anyone wants it.

 

More interestingly though, I've discovered you can set a flag in the bink videos to make them play at double size. Go 36 bytes from the start of the bik header and change 0x00000000 to 0x00000040. If you've patched your exe to set the resolution to 1280x960 (or higher, but 1280x960 is best for the backgrounds and things) then this is really great . The subtitles stay in their original location, which looks a bit odd, but if you really want you can disable them completely by removing the smush header (seems disabling subtitles in the settings makes the video not play at all on current versions of windows).

[Benny]

Great find! That'll really help enhance the resolution patch!

[s-island]

Also, the PS2 video is not cut off at the bottom as I said earlier but simply squished a little

Are you playing the PAL or NTSC version? This is a typical artifact that comes from porting games from NTSC to PAL on the cheap. PAL has about 100 extra lines of resolution so instead of re-rendering everything, they just put the original NTSC picture in the middle, leaving about 50 black lines on the top and bottom.

[Longcat]

Great find! That'll really help enhance the resolution patch!

 

 

Seeee-weeeeeet!

[Benny]

Gannet's delving into EMI also goes beyond this thread. He's sent me a fix so that windowed mode now works when EMI is patched into a higher resolution.

As a result I'm updating my EMI launcher and its resolution patch. It'll include the windowed fix, Gannet's 'double size' video patch and will let you choose a custom resolution, rather than the 4 pre-selected ones there are now.

 

The reason I'm posting this is because I also want there to be a decent list of common resolutions to choose from for non-expert users. There'll be the option to type in your own res, but I want to keep the dropdown list with the resolutions already in. Does anyone have a list of common resolutions that you think should be included? I don't want there to be so many that the choice is overwhelming, but there should be a reasonable selection for 4:3, 16:10 and 16:9 aspect ratios.

[Longcat]

as long as there are 16:10 and 16:9 options, I'm happy. don't know about netbooks though, they sometimes have weird aspect ratios, but I guess they are not in the target audience. thanx for updating the patch bg, I'm really looking forward to playing this again at a reasonable resolution. I've stayed clear of the patch before because the videos still would play in a small window. and I love the idea of having a custom resolution input:)

 

Thank you both!

[Gannet]

@s-island: Hm, I'm not sure. But I suspect it's to do with subtitles rather than an NTSC/PAL thing - the .til files are still 640x480.

 

@bgbennyboy: Put emphasis on 1280x960 - the videos are fullscreen, the backgrounds scale the nicest and the save thumbnails display properly. For different resolutions though, remember the fix to black out the background when a video is playing. Thanks heaps

 

Still looking for tools to work with the voice files...

[Benny]

Gannet: I've emailed you about the 'black the background out' fix - I couldn't get it to work.

 

I'm not sure what info there is on the voice files floating around, from what I remember I think they use some sort of compression (VIMA?) in a wav container or something similar. I'm pretty sure that John_Doe's EMI Resource Viewer and one of the versions of SCUMM Revisited could play them back.

[Gannet]

Hrm, haven't seen any email from you :/

Offset 0x22F9E reads 0x8002. It works for me if I change this to anything else (doesn't matter what the resolution is).

 

Scumm Rev 5 can extract the voice files but not play them. v3 download no longer works. EMI Resource Viewer 2.1 doesn't play them either and v2.6 download no longer works.

[edit] Okay, I may not be able to play the files but I can at least read the header. Both the Windows and PS2 voice files are 16538Hz 16-bit mono but the PS2 files are smaller than the Windows ones, implying more compression. So I can't see how the PS2 voice could be better.

[Benny]

EMI Viewer 2.6

 

I've dug out my PS2 copy of EMI and its a German PAL version. I can upload some files if you need to compare.

 

I tried changing 0x8002 to other values at that offset - none seemed to make any difference - the screen isn't blacked out. Are you sure that's the only value you changed?

 

[Edit] I've had a quick look at the PS2 voice files, I havent done anything more than just look at them in a hex editor but my guess is that they use the PS2 VAG format, which is an adpcm variant.