Some Lua details

[Mike.nl]

As everyone knows, the Empire at War Lua files are some weird Lua variant.

Some while ago, I figured out what exactly made these lua files 'weird':

1. The signature is "\033Lup" instead of "\033Lua"
   
2. The indicated version is 0x51 instead of 0x50 (yes, the files are actually in the 5.0 format)
   
3. Every function contains an additional integer, after the "lineDefined" integer. I don't know what this integer is for, but it doesn't seem to serve any real purpose when reading the files.
   

 

So, read a EaW Lua file (or, as I like to call them thanks to their signature: Lup files), change the signature, change the version and remove the extra integers and you're left with a perfectly valid Lua 5.0 file.

 

Unfortunately, this is a long way from the Lua source code. I am currently aware of two decompilers: LuaDC and Luadec. Neither of which succesfully decompiled the files.

 

LuaDC complained about invalid filenames and what not when I started it, and without the source I have no idea what's wrong with it (and I didn't give up easily).

Luadec was able to read the files but it crashed on decompilation. As it turned out (yay for open-source), luadec assumes the information about a function's locals is present, whereas it is, in fact, not (the locals information is optional, and Petroglyph apparently decided to strip that. Can't blame them).

 

After some l33t hacking in luadec I got to work around this by tricking it into thinking it had locals information. Now it actually produced Lua code from the most basic of Lua files. However, when decompiling the more complicated Lua files luadec is unable to make sense of them, leaving me less then impressed with this decompiler.

 

I've even gone so far as to take a pot shot at writing a decompiler myself, but proper decompilation is hard, even for a 'simple' language such as Lua. Well, it is for me anyway.

 

On the other hand, I can of course output Lua assembly from the lua files; that's easy to do. But then this has to be manually decompiled into normal Lua code, something I'm sure most modders can't, won't (or even shouldn't) do.

 

Anyway, I just thought I'd put this information on the forums. Maybe someone who does know a thing or two about decompilation can use this info, or perhaps point me to, or adapt, another existing decompiler.

 

 

Update 2006/05/05: decompiled lua files are now available: http://alpha1.dyns.net/eaw/

[Orao]

If you could figure out this one it will save us alot of trouble especialy with the AI.

 

Thank you for you findings so far. COntinue the good job.

[Orao]

BTW I just forgot to ask you. What files do you edit for scripted action on the welcome screen ?

 

Thank you.

[Admiral Sith]

Thats in the gameconstants.xml, its actually a replay of your most recent recorded game, a feature that was, unfortunatly, removed. So you get stuck with that same one.

[Slocket]

Well Mike good job so far! This is the REAL editor I am looking for. Can you give us a version of the LuaDec you modified?

 

On the other hand, I can of course output Lua assembly from the lua files; that's easy to do. But then this has to be manually decompiled into normal Lua code, something I'm sure most modders can't, won't (or even shouldn't) do.

 

 

I programmed way to much in assembler decades ago when computers were very 'dumb'. Even the decompiled asssembler code can be of great use to do basic modification of integer -say what the AI will use for a raid fleet total number or what it thinks it needs to attack your fleet.

 

Then by going into the regular files, it can be hexed in. I did this for the DLL in Freelancer to get around some annoying behaviour to allow playing the game in single player mode without the storyline getting into the way.

 

So if you can give out a copy, I think quite abit of simple modding of the scripts can be done to some extent using the assembler code as a guide.

 

This well give the game a boost by making the AI smarter.

[Slocket]

Hey, the old demo is written slightly different than the final game release Lup. I manage to follow you advice for the old demo file Lua 5.0 and it decompiled quite a bit until it crashed! lol

 

I thought you may find this interesting. I only used LuaDec on BuildTurboLaser.lua of the old demo. I change the header from Lua(51) to Lua(50). No other change yet and it decompiled the bugger until it hit the 'end' or that integer thingy. The other thing it seems to have a problem with handling End of Line.

 

If you can show me what you did with the LuaDec I think with this information; you will have a decompiler of some sort.

 

BuildTurboLasers.lua output before crash:

 

----------------------------------------------

1 GETGLOBAL 0 0

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{require}

 

----------------------------------------------

2 LOADK 1 1

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 0{require} 1{"pgevents"}

 

----------------------------------------------

3 CALL 0 2 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

require("pgevents")

 

----------------------------------------------

4 CLOSURE 0 0

----------------------------------------------

1 LOADK 0 1

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{"Build_Turbolasers"}

()

 

----------------------------------------------

2 SETGLOBAL 0 0

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(1): -1{Category="Build_Turbolasers"}

tpend(0):

()

 

----------------------------------------------

3 LOADBOOL 0 1 0

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{true}

()

Category = "Build_Turbolasers"

 

----------------------------------------------

4 SETGLOBAL 0 2

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(1): -1{IgnoreTarget=true}

tpend(0):

()

Category = "Build_Turbolasers"

 

----------------------------------------------

5 NEWTABLE 0 1 0

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

6 NEWTABLE 1 2 0

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

7 LOADK 2 4

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 2{"StructureForce"}

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

8 LOADK 3 5

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 2{"StructureForce"} 3{"Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

9 SETLIST 1 1

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 1{{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

10 SETLIST 0 0

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{{{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}}

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

11 SETGLOBAL 0 3

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(1): -1{TaskForce={{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}}

tpend(0):

()

Category = "Build_Turbolasers"

IgnoreTarget = true

 

----------------------------------------------

12 RETURN 0 1 0

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Category = "Build_Turbolasers"

IgnoreTarget = true

TaskForce = {{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}

 

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{function()

Category = "Build_Turbolasers"

IgnoreTarget = true

TaskForce = {{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}

end

}

require("pgevents")

 

----------------------------------------------

5 SETGLOBAL 0 2

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(1): -1{Definitions=function()

Category = "Build_Turbolasers"

IgnoreTarget = true

TaskForce = {{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses = 1"}}

end

}

tpend(0):

require("pgevents")

 

----------------------------------------------

6 CLOSURE 0 1

----------------------------------------------

1 GETGLOBAL 0 0

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{Sleep}

()

 

----------------------------------------------

2 LOADK 1 1

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 0{Sleep} 1{1}

()

 

----------------------------------------------

3 CALL 0 2 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

 

----------------------------------------------

4 GETGLOBAL 0 2

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{StructureForce}

()

Sleep(1)

 

----------------------------------------------

5 GETTABLE 0 0 253

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{StructureForce.Set_As_Goal_System_Removable}

()

Sleep(1)

 

----------------------------------------------

6 LOADBOOL 1 0 0

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 0{StructureForce.Set_As_Goal_System_Removable} 1{false}

()

Sleep(1)

 

----------------------------------------------

7 CALL 0 2 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

 

----------------------------------------------

8 GETGLOBAL 0 4

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{AssembleForce}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

 

----------------------------------------------

9 GETGLOBAL 1 2

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 0{AssembleForce} 1{StructureForce}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

 

----------------------------------------------

10 CALL 0 2 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

 

----------------------------------------------

11 GETGLOBAL 0 2

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{StructureForce}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

 

----------------------------------------------

12 GETTABLE 0 0 255

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{StructureForce.Set_Plan_Result}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

 

----------------------------------------------

13 LOADBOOL 1 1 0

SET_CTR(Tpend) = 1

 

next bool: 0

locals(0):

vpend(0):

tpend(2): 0{StructureForce.Set_Plan_Result} 1{true}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

 

----------------------------------------------

14 CALL 0 2 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

 

----------------------------------------------

15 GETGLOBAL 0 6

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{ScriptExit}

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

 

----------------------------------------------

16 CALL 0 1 1

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

ScriptExit()

 

----------------------------------------------

17 RETURN 0 1 0

 

next bool: 0

locals(0):

vpend(0):

tpend(0):

()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

ScriptExit()

 

SET_CTR(Tpend) = 0

 

next bool: 0

locals(0):

vpend(0):

tpend(1): 0{function()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

ScriptExit()

end

}

require("pgevents")

Definitions = function()

Category = "Build_Turbolasers"

IgnoreTarget = true

TaskForce = {{"StructureForce", "Q_Galactic_Turbolaser_Tower_Defenses | X_Galactic_Turbolaser_Tower_Defenses =

[Mike.nl]

I've written a small (command-line) tool that will convert Lup files to Lua files and back: lup2lua (and its source).

 

Second, here are my (small) modifications to luadec: luadec-0.6-mod.

I only changed two things in the original file, and those changes are marked by my name, so you should find them easily. Running the resulting luadec over BuildTurboLasers.lua results in:

 

C:\>move BUILDTURBOLASERS.LUA buildturbolasers.lup

C:\>lup2lua buildturbolasers.lup buildturbolasers.lua

C:\>luadec buildturbolasers.lua

 

require("pgevents")

Definitions = function()

Category = "Build_Turbolasers"

IgnoreTarget = true

TaskForce = {{"StructureForce", "E_Galactic_Turbolaser_Tower_Defenses | R_Galactic_Turbolaser_Tower_Defenses = 1"}}

end

 

StructureForce_Thread = function()

Sleep(1)

StructureForce.Set_As_Goal_System_Removable(false)

AssembleForce(StructureForce)

StructureForce.Set_Plan_Result(true)

ScriptExit()

end

 

StructureForce_Production_Failed = function(L0, L1)

DebugMessage("%s -- Abandonning plan owing to production failure.", tostring(Script))

ScriptExit()

end

 

Yet, running it on larger, more complex lua files will crash the decompiler as usual. I've tried to find out why, but I don't understand the laudec code that well and it seems like more than just a superficial problem.

 

Anyway, I'll create a lua disassembler over the next days, if that'll help you guys.

[Slocket]

Sure, please do. I do not think we will get a LUA editor for now. With a disassembler, we can (the hard way) change any small aspect of the game that many modders may like.

 

I for one would like to change the way the DeathStar Battle is played out. If it means to manually recreating the script from assembler -so be it. It would only be a few hours work to type up the text script. I assume the EXE will read the expanded form of it.

 

Thanks for all your hard work Mike!

[squeegee]

Anyway, I'll create a lua disassembler over the next days, if that'll help you guys.

 

I had just used winhex to update the sig bytes before I found out about the extra integer's and thus luadec's failure to decompile. Decided to google at that point, and glad I did!

 

I've written several decompilers over the years, so I'll see what I can figure out about the crash with luadec.

 

Thanks for the effort you've put into this! What an awesome place to start!

[popcorn2008]

Good work Mike! It would be great to have a community made LUA deassembler until/if Petro releases their own.

[Foshjedi2004]

LoW's Coder Alisama is busy doing Model File Comparisons So We may have a Filefront Community Model Converter and Bone Editor

[squeegee]

I figured out the luadec crash: It's because of the lack of local information.

 

I coded around it by using _TEMP_ for all local variable names. At least that got me able to decompile everything I've tried it on so far, but make no mistake, it's a kluge (for example, if there are 5 locals, all 5 are named _TEMP_ - not good!).

 

Email me if you want a copy of the modified source (no place to put it online...).

 

I will work on a proper solution to the local variable issue next. I just have to do some heavy code reading to understand luadec parse structure...

[Athanasios]

I figured out the luadec crash: It's because of the lack of local information.

 

I coded around it by using _TEMP_ for all local variable names. At least that got me able to decompile everything I've tried it on so far, but make no mistake, it's a kluge (for example, if there are 5 locals, all 5 are named _TEMP_ - not good!).

 

Email me if you want a copy of the modified source (no place to put it online...).

 

I will work on a proper solution to the local variable issue next. I just have to do some heavy code reading to understand luadec parse structure...

 

Can't you set a generator for random naming of local variables?

 

I have no clue of LUA coding, but in C, C++, Java and such this is doable. Either with matrices of strings or pointers.

[squeegee]

Can't you set a generator for random naming of local variables?

 

I have no clue of LUA coding, but in C, C++, Java and such this is doable. Either with matrices of strings or pointers.

 

Actually, my original kluge was pretty stupid. As it turns out, all the crashes I've come across are related to the local vars. All I had to do was before using a local variable check for it being null, and if it was, call the function Mike created. The only other change was to make sure not to free the auto locals on function closure.

 

So, now, all local variables have distinct names of L0, L1, L2, etc. The best we can do without the local variable information (i.e. not knowing the actual local variable names). The code is quite readable, and I have successfully decompiled all the lua code in config.meg now.

 

There is one other strange bug in luadec: When a function is declared, the line of code generated is like "name = function()" instead of "function name()". It's annoying, but the code is still readable. I'll look into eventually...

 

I can make a zip of all the decompiled sources if anybody is interested. As well as the fixed luadec and code if interested.

[popcorn2008]

If you could do us the favor and make a zip of the decompiled scripts and modified luadec it would be great.

 

It could give us a head start on using Lua.

[Athanasios]

I would suggest to work closely with Mike, so that you break Lua till the end. So that we can even compile models too.

 

Yet, in the meanwhile, you can upload what's you've done. Just mind to post this to PFF forums too, there are people interested for this there too!

[squeegee]

I would suggest to work closely with Mike, so that you break Lua till the end. So that we can even compile models too.

 

Yet, in the meanwhile, you can upload what's you've done. Just mind to post this to PFF forums too, there are people interested for this there too!

 

Agreed, just waitng for a response. luadec has some internal problems, so I am looking into the actual structure of tokenized lua now to see if it's easier to start over or attempt to fix luadec.

 

From my perspective, luadec is more complicated than it needs to be (it appears as though it actually attempts to walk the code by pseudo-executing it, which is, uh, not the way I'd do it...).

 

As I mentioned before, I've written several deompilers so I am quite familiar with how to get one up and running quickly, and correctness is the most important thing for a decompiler IMO, and luadec still bails on some simple valid structures (which I just noticed the failure messages in a few of the files I decompiled - although 13 failures out of 260 isn't _that_ horrible for our purposes).

[squeegee]

Ok, I've found what I consider the fundamental issue with luadec: It's not decompiling the binary code directly, but using Lua to load and walk the code. As a result, the internals are quite ugly, as a lot of Lua specific internal states, upvalues, etc. all have to be tracked.

 

The odd thing is that none of that is needed. The object format is very straight forward, and the mapping from the 35 opcodes to the source is almost direct. Even without the debugging info (function names, local variable names), recreating the original source should be doable.

 

Anyway, I've been studying chunkspy as well (a disassembler for Lua object code writen in Lua - only 2100 lines, with lots of comments), and I think I am going to work on creating something from scratch over the weekend. I'll post the progress.

 

This is so much more fun than working on work! :-)

[Athanasios]

Squeegee, mind that decompliling is one step, next step will be re-compiliing our modded lua files. As for opcodes, you should pay very attention so that we dont mess with micromemory (virtual i guess?), but you already know this .

 

If you can do both of these, then you should be given the "golden EaW metal" and ill be the first one to contact admins for this !

 

As soon as we can get our hands dirty with lua scripting.........our way is open .

 

Keep up the good work!

[Mike.nl]

Allright, this is it!

 

I have finished two tools that:

• Converts between the official Lua format and Petroglyph's Lua format.
  
• Disassembles a Lua 5.0 object file.
  

I used these tools (among others) to decompile most Lua files (91%, lacking some story scripts for now).

 

Yes, that's right. Lua sources are available.

Along with instructions on how to create custom Lua files for Empire at War.

Get it all here: http://alpha1.dyns.net/eaw/

 

Lacking a good decompiler, I've manually decompile everything luadec crashed on, and I still have to do the remaining story scripts, but I decided to release the Luas I've decompiled so far so you guys have something to work on

 

Of course, keep in mind the source is uncommented and uses a bit vague function arguments, which is inherent to any decompilation. Perhaps I will one day rename them to something meaningful, based on what they do, once I have time.

 

Anyway, as usual, if you have questions or remarks (both the good and the bad kind), post here.

[squeegee]

Allright, this is it!

 

I have finished two tools that:

• Converts between the official Lua format and Petroglyph's Lua format.
  
• Disassembles a Lua 5.0 object file.
  

I used these tools (among others) to decompile most Lua files (91%, lacking some story scripts for now).

 

Kewl.

 

Yes, that's right. Lua sources are available.

Along with instructions on how to create custom Lua files for Empire at War.

Get it all here: http://alpha1.dyns.net/eaw/

 

Lacking a good decompiler, I've manually decompile everything luadec crashed on, and I still have to do the remaining story scripts, but I decided to release the Luas I've decompiled so far so you guys have something to work on

 

Ouch. Thanks for taking the time. I also have decompiled all but the 13 files luadec ended up bailing on (I fixed some of the crahes to get that far).

 

Of course, keep in mind the source is uncommented and uses a bit vague function arguments, which is inherent to any decompilation. Perhaps I will one day rename them to something meaningful, based on what they do, once I have time.

 

Anyway, as usual, if you have questions or remarks (both the good and the bad kind), post here.

 

I am working on a good decompiler. It can decode the binary files for both debug and non-debug already (like what -list can do with luac), so today I am starting the source code generation.

 

Unlike luadec, this loads the binary object directly, wihtout Lua's involvement. The code is only 500 lines of c++ so far. I should have the source generation working reliably within another couple of days. I've named it luad (I know, how original...).

 

Once this is done, along with your lup2lua tool, and luac, we should be able to decompile, modify, and recompile the sources!

 

If you want, we can combine all of the tools into one, so it can be done easier for the non-tech types out there.

 

Great job!

[Athanasios]

MUST BE STICKIED I-M-M-E-D-I-A-T-E-L-L-Y !!!

 

Also Mike, do not forget to post this over PFF forums too. The people will go mad!

 

And someone give those guys a golden medal . Mike, i suggest you work with squeege so that we get 100% bug-free and 100% commented lua files.

 

Check your pm also .

 

[edit] I posted this for you in PFF, just in case you're not registered .

[Mike.nl]

Mike, i suggest you work with squeege so that we get 100% bug-free and 100% commented lua files.

 

I'm pretty sure the code is bug free. I recompiled my decompiled code and the resulting object files was an exact binary match with the original.

 

And we wouldn't want 100% commented lua files. That would leave no room for the actual code

 

I posted this for you in PFF, just in case you're not registered .

Yeah, I'm not, so thanks

 

Actually, the biggest challenge now is to create documentation. Establish an API. Find out how the Luas interface with the game and how far that interface can be stretched.

 

 

And to squeegee:

I'm looking forward to your decompiler. It would certainly be nice to bundle these tools.

Out of curiosity, how robust will your decompiler be? Will it depend heavily on known constructs by the compiler? Or is it able to generate equivalent source code from other, but functionally equivalent code as well?

For instance, the compiler compiles while loops with the test at the end, but what if the test were moved to the beginning? And what if the assembly was more optimized in terms of register (re)usage? The standard lua compiler is pretty brain-dead from what I can tell.

 

But I do agree, the lua object format is ridiculously simple, that's why luadec never impressed me that much, with all its bloat.

[squeegee]

And we wouldn't want 100% commented lua files. That would leave no room for the actual code

 

Comments aren't stored in the object files, so it wouldn't matter. I'm sure you know this, just making sure everyone else does.

 

 

Out of curiosity, how robust will your decompiler be? Will it depend heavily on known constructs by the compiler? Or is it able to generate equivalent source code from other, but functionally equivalent code as well?

For instance, the compiler compiles while loops with the test at the end, but what if the test were moved to the beginning? And what if the assembly was more optimized in terms of register (re)usage? The standard lua compiler is pretty brain-dead from what I can tell.

 

It will recreate the original source as close as possible; It will generate the exact same object, or it's not done (I mentioned before that IMO a decompiler's most important trait is accuracy).

 

The lua compiler (tokenizer...) doesn't need to optimize much, as the vm assembly is so simple, and the vm itself can optimize stuff out. If you read some of the vm asm specific stuff out there, they point these things out, as well as other quirks. Most of those doc's are dated in the last 6 to 9 months, so they didn't exist when luadec was written.

 

My "plan" is to be able to decompile any lua source. Once that works perfectly, then I'll derive a version specific to EAW/Lup, and we can combine the stuff into a single executable. Maybe even provide a simple GUI so non-techies can use it (of course, maybe they shouldn't be modifying the code anyway... ;-).

 

But I do agree, the lua object format is ridiculously simple, that's why luadec never impressed me that much, with all its bloat.

 

Luadec's issue is the writer evidently hasn't written anything of the sort before, and attempted to use Lua's code to do some/most of the work.

 

What I have is entirely stack based, and thus has no limits on depth of functions, loops, etc. (well, other than the memory of the machine it's running on). By definition, it can only re-generate what the object code contains. That is, if the compile moves things around (which I haven't seen a case of yet, btw), then it will regenerate it that way, as there is no way to know different. Functionally, though, it'd be the same as the original, and the object code will be the same.

 

Looking forward to getting this done; It's been a lot of fun so far!

 

BTW I can send the code to anyone who wants it at any time. It'll be public domain or BSD/MIT styled licensed when I get done with it.

 

Also, Mike, if you want the version of luadec that has fixed most of the crashes, let me know. I have a site I can put it on, I just have to put some work into the web site first (the domain is for email, but the http hosting is included but currently unused).

[Slocket]

A big thank you to you guys! You know your stuff very well (this is way above my head). It will be good to have even more versions to work with, in case something came up for redunancy.

 

As for me, the 'technically challenged', I can play with the AI game mechanics and see what happens! lol