Need help by someone 2da and tlk knowledgeable

[stoffe]

Background:

When I was working on my new force powers mod (and experimenting with adding new classes), it annoyed me that there was no easy way to add new names and descriptions save distributing a whole dialog.tlk file, Likewise I could find no easy way to make it somewhat compatible with already modified 2da files in override.

 

So a few days ago I started working on a small "patcher" application that will append new strings to dialog.tlk, and update existing 2da files (like ones in the override folder) with the correct StrRefs and new rows. This "patcher" will also let you add new lines and columns to 2DAs and change existing values in 2DA files.

 

Anyway, this application seems to be working fairly well now as far as I can tell, but since it's easy to become blind to your own mistakes I'm looking for some brave person/people who would be willing to help me test it and play a bit with this patcher app and see if it works as it should. Any takers?

 

 

 

(Hmm, I believe this is the first topic I have started on this forum, so don't kill me if this is concidered off-topic... )

[Achilles]

Count me in. This would actually work perfectly for a mod that I made, lost in the crash, and wasn't looking forward to rebuilding.

[RevanA4]

I'm game

[Tupac Amaru]

I'd be happy to help if three isn't too much.

[T7nowhere]

I'll help test it out.

[ChAiNz.2da]

Someone say ".2da" ??

 

I'd be happy to help, though to be honest, I've never dealt with the .tlk file.. but .2da's are right up my alley

[envida]

Count me in, I'd be happy to help

[RedHawke]

*Waives hand in air*

 

Me too stoffe!

[tk102]

Welcome Developer!

 

 

I'm looking forward to this release. Be careful or you might find utility-making is more fun than mod-making, just as mod-making is more fun than game-playing.

 

Hope you'll choose to publish your source code too.

[Darkkender]

Did somebody say new utility? I'm in. Of course I'm curious stoffe I know you've been playing with adding new columns to 2da files will your utility implement this functionality?

[stoffe]

Originally posted by tk102

Be careful or you might find utility-making is more fun than mod-making, just as mod-making is more fun than game-playing.

 

Hope you'll choose to publish your source code too.

 

Hopefully, even though programming tends to involve plenty of swearing on my part as well, it's worth it when it finally starts to work as intended.

 

I suppose I could put up the source too when it is finished, though it'll probably not be fun reading since the code is rather messy. I hadn't done any real programming for 2 years before I started on this, so I was a little rusty when I started.

 

(And the app is written in Delphi so you probably won't have much use for the source unless you have Delphi, other than to point and laugh at my poor programming skills. )

 

Originally posted by Darkkender

Did somebody say new utility? I'm in. Of course I'm curious stoffe I know you've been playing with adding new columns to 2da files will your utility implement this functionality?

 

Yes, the patcher can currently add new columns to 2da files. Should save plenty of annoying spreadsheet import/export work when playing with adding new classes

 

----

 

I should perhaps add, since I got that question in PM, that I plan to add GFF file patching to the mix as well, so newly added 2da lines can be properly referenced from Item blueprints and such.

 

This may take a little while though since the GFF format is a fair bit more complex than TLK and 2DA. But I'm working on a GFF-read/write class that will let the patcher modify GFF files.

 

I've only been working on this app since last monday so far, so I think it's best to verify that the TLK/2DA patch functionality works first before I add something as complex (for me anyway) as GFF support to it though.

 

----

 

I should perhaps also add that while some error checking is in place it is by no means foolproof at this point. When editing the changes.ini file, be careful when you create new modifier lists or keys/values. That said, if you do make mistakes that cause problems, please let me know so I can improve the error checking and make the patcher safer to work with.

 

Please post about any other bugs you encounter as well and I'll look into it. If you have any suggestions those are welcome as well.

 

 

(PMs with the patcher app in its current form has been sent to a bunch of you who were willing to help me test it. Check your inboxes.)

[General Kenobi]

Sounds Goooood Skoffe If you need testers and the like give me a hollar If graphics are an issue I can most def. help with them if you need em' at some poiint. Just PM and let me know if you want me to test it for ya.

 

DM

[Darkkender]

That is great that it will support new columns stoffe I'll start testing that end of things ASAP.

[stoffe]

I have put up a slight update of the Patcher, same URL as before. This new version fixes a whole load of bugs and problems I encountered.

 

Among other things It will now refuse to add columns with the same label as one that already exists. I also fixed some GUI problems, such as window resizing, and the Quit button no longer staying dimmed if aborting due to errors.

 

The error/problem/info reporting has also been somewhat improved. (Set the log level to 4 in the INI file to see most of it.)

 

 

It also has 4 new features. Though nothing earth-shattering, they are:

 

1) Will search dialog.tlk for an identical entries to those in append.tlk. If encountered it will use the existing entries when using StrRef tokens to set values in 2DA columns, and not append the matching lines to dialog.tlk. No point in adding duplicates.

 

2) Added optional option in the ini file to set the text caption of the patcher window.

 

3) Two new "special" values are added to the CopyRow modifier. They are:

 

inc(#) - Increments the value in the column it is assigned to by # for the newly copied row. Only works on columns containing numbers.

 

high() - Finds the highest value on any row in the column it is applied to, and then set the value of that column to that value +1 for the newly copied row.

 

4) I've added a limited memory of sorts. In any sections in changes.ini tied to a AddRow,ChangeRow, AddColumn or CopyRow modifier you can now assign any column label (or row label in the case of AddColumn) to a special key called 2DAMEMORY#. The value in the 2DA of that cell is then temporarily stored.

 

 

You can then use this stored value anywhere in those modifier sections by assigning it to another column (or row) label.

 

Here's a theoretical example to demonstrate how this could be used... The following work instructions would add a new force power based on the row of Force Kill, learnable at lvl 21. It would then modify Force Crush to be learnable at level 23 for a Sith Lord, but add the newly added Force Power as a prerequisite for it.

 

[TLKList]
StrRef0=0
StrRef1=1

[2DAList]
Table1=spells.2da

[spells.2da]
CopyRow1=spells_forcegrip
ChangeRow1=spells_crush

[spells_forcegrip]
RowIndex=32
2DAMEMORY1=RowIndex
label=FORCE_POWER_GRIP
name=StrRef0
spelldesc=StrRef1
guardian=21
consular=21
sentinel=21
weapmstr=21
watchman=21
jedimaster=21
marauder=21
assassin=21
sithlord=21
iconresref=ip_grip
impactscript=st_fp_grip
forcehostile=high()
dark_recom=****
pips=4

[spells_crush]
RowLabel=177
prerequisites=2DAMEMORY1
sithlord=23

 

(This would assume that append.tlk contains two entries with StrRef 0 and 1:

0: Name of the power

1: Description of the power

...in order to work.)

 

Check the changes.ini file, I've updated the comments with some info about these additions.

 

---

 

(I've also started working on adding GFF support to the patcher, but it'll probably take a week or two to get that working. It's not present in the new version I uploaded.)

 

 

EDIT: Did another small update, changing it to use multiple, number-indexed Memory Key slots instead, to make it more capable of linking together new rows in different 2da's that reference eachother. Included an example.ini that adds a new appearancce/head/portrait to demonstrate how it can be used to do that.

[stoffe]

I found a nasty bug/oversight with the Memory tokens that caused them not to work properly when assigned to in a AddColumn operation. Sorry about that, I've uploaded a new version of the patcher that fixes this.

 

I also added support for the dual dialog.tlk + dialogf.tlk thing that some foreign language versions use, that I'd forgotten about before. If you add an appendf.tlk file in the tslpatchdata folder the Patcher will ask for the dialogf.tlk file when run and apply those entries to it.

 

 

I know you probably are busy with more important things, but has anyone had time to test the patcher yet? If so, does it seem to work ok?

 

(If anyone has a non-english version of the game I'd also be interested to hear if the above mentioned dialogf.tlk support works as it should. Since I have the UK version of the game I can't test that myself...)

[Achilles]

The only thing that I've noticed thus far is that when I updated spells.2da it made Force Crush a selectable force power. I'll dl the updates and see if it's still an issue.

[stoffe]

Originally posted by Achilles

The only thing that I've noticed thus far is that when I updated spells.2da it made Force Crush a selectable force power. I'll dl the updates and see if it's still an issue.

 

That sounds like one of the example modifiers I left in the changes.ini file to make it (hopefully) a bit easier to understand how it fits together.

 

Remove the line "ChangeRow2=spells_crush4all" from the spells.2da modifier list in changes.ini and that should fix it.

 

You should probably toss out all the existing modifiers in that file and write your own, since they are only there as examples that don't really do anything meaningful.

 

Hmm, I should perhaps elaborate a bit more about the structure of that file, since it probably isn't so clear for someone who hasn't created it.

 

(You may stop reading here if you aren't interested in my very long rambling explanation which will occupy the rest of this post. You have been warned. )

 

----

Like all INI files, changes.ini is divided into sections. Each section has a section identifier enclosed in [square brackets]. Under each section is a series of Key=Value specifiers. There are four main sections that are always named the same. They are:

 

[settings] - contains some settings for the patchet itself. You can change some of the text in the GUI here, and you can set how detailed the feedback text the patcher displays should be.

 

* * *

 

[TLKList] - a table listing the custom StrRefs in append.tlk that should be added to the dialog.tlk (and dialogf.tlk if present) file. The Key part (the text to the left of the "=" character) is also usable as a Token that you can use when modifying 2DA files to insert the correct StrRef to that table cell.

 

The format is

Token=StrRef

like

StrRef0=0

StrRef1=1

StrRef2=320

...and so on, one for each string that should be added.

 

If the patcher shouldn't add any new strings, this section should be left out entirely.

 

* * *

 

[2DAList] - lists the NAMES of all 2da files that should be modified by the patcher in the format:

Table#=name

...like:

[2DAList]
Table1=heads.2da
Table2=appearance.2da
Table3=portraits.2da

 

Note that if the 2da files you want to modify reference eachother, the ordering may be important since the modifications are executed in the order they are listed.

 

The above order would for example be necessary to add a new player appearance since the new heads.2da line needs to be referenced by the new appearance.2da line which in turn needs to be referenced by the new portraits.2da line.

 

* * *

 

(NEW)

[GFFList] - lists the NAMES of all GFF-formatted files (such as UTI, UTC, UTP, UTM, DLG files etc..) that should be modified by the patcher. The format is

File#=filename

...like:

[GFFList]
File1=p_bastila.utc

 

The File# key names must be unique for each file, since no two keys in a section can have the same label.

Tied to this is a section named the same as the filename for each file, which lists the changes that should be made to that file in the format:

fieldlabel=new value

...like

[p_bastila.utc]
Appearance_Type=2DAMEMORY1
PortraitId=2DAMEMORY3

 

This would insert two stored values (2DAMEMORY1, 2DAMEMORY3) into the fields Appearance_Type and PortraitId in the file named p_bastila.utc. (More about the 2DAMEMORY tokens below.)

(/NEW)

 

* * *

 

The file then contains a number of sections with dynamic names, which are linked to from the 2DAList section. Each 2DA file listed in the 2DAList section has a section named after it, which contains a "modifier list" of instructions on what should be done with that file. To add to the above example you would have these sections:

[heads.2da]

[appearance.2da]

[portraits.2da]

Under each of those sections follow a list of operations to perform on that file. They use this format:

 

Action=Instructions

...Where Action is one of the following:

 

AddRow#

ChangeRow#

CopyRow#

AddColumn#

 

The # part is a unique sequential number, as no section may contain two keys named the same.

 

The Instructions part is the name of another section in the file, which will specify what changed to make. You can name these section identifiers whatever you wish, so name it something that will help you remember what it's for.

 

So to sum up, to add a new row to each of those three 2da files, based on another existing row, you would get something like:

[heads.2da]
CopyRow1=copy_heads

[appearance.2da]
CopyRow1=copy_appearance

[portraits.2da]
CopyRow1=copy_portrait

* * *

 

Next would follow the sections you've referenced on the 2da-specific sections above. Thus you would have these new sections:

[copy_heads]

[copy_appearance]

[copy_portrait]

 

For a section attached to a CopyRow modifier, as in our example, each section must start with either a "RowIndex" key or a "RowLabel" key, which will tell the patcher which row to use as a template to copy.

 

Following that comes an optional "NewRowLabel" key. This is only necessary for 2DAs which don't have row labels that match their row index, like most tend to. Visualeffects.2da is one such example. If left out, the RowLabel will be set to the same as the RowIndex (ie line number) of the new row.

 

The rest of the keys in that section is then expected to be column labels from the 2da file. The value part will be assigned to that column for the new row copy.

 

There is one extra type of key that can be used, which has a special purpose. You can use it to store a value from the column whose label you assign to it, which can then be recalled and assigned to another column elsewhere. This key is 2DAMEMORY#, where # is a number, starting at 1 and going up, that designate which memory "slot" should be used. Hopefully this will be more clear in the example below.

 

To continue with the above example, let's modify those three 2da files to add a new female player appearance based on Bastila's lines. You would then get instruction sections something like:

 

[copy_heads]
RowIndex=1
2DAMEMORY1=head
alttexture=2DAMEMORY1
headtexvvve=P_BastilaHD2
headtexvve=P_BastilaHD1
headtexve=P_BastilaHD1
headtexe=P_BastilaHD1
2DAMEMORY2=RowIndex

 

This would copy line number 1 (Bastila's head) in heads.2da and add the new copy at the end of the file. It would then put the current value in the "head" column into Memory Slot 1, and assign that value to the "alttexture" column as well. It would then add Resrefs to two new darkside head textures for the head, since those columns were "****" in the original line. Finally, it will store the row index of the NEW LINE in Memory Slot 2.

 

[copy_appearance]
RowIndex=4
label=P_FEM_C_BASTILA
driveanimwalk=2.6
NormalHead=2DAMEMORY2
modela=PFBAM
texa=PFBAMC
texaevil=PFBAMD
modeli=PFBIM
texi=PFBI
modell=PFBL
texl=PFBLC
texlevil=PFBLCD
modeln=PFBNM
texn=PFBN
2DAMEMORY1=RowIndex

 

This will copy line number 4 (Bastila's appearance type) in appearance.2da and add a new line identical to it at the end of the file. It will update the label to identify our new appearance type, and will assign the value in Memory Slot 2 to the NormalHead column (which was the Row Index of the new line in heads.2da we just added and saved above). Further it will fill out the blank body variation columns with appropriate values, and finally it will store the Row Index of this NEW LINE in Memory Slot 1.

 

[copy_portrait]
RowIndex=33
appearancenumber=2DAMEMORY1
appearance_s=2DAMEMORY1
appearance_l=2DAMEMORY1
forpc=1
baseresrefe=po_bastilad1
baseresrefve=po_bastilad1
baseresrefvve=po_bastilad1
baseresrefvvve=po_bastilad2
2DAMEMORY3=RowIndex

 

Finally, this will copy line number 33 (Bastila's portrait line) in portraits.2da and add the newly copied line at the end of the file. Next it assigns the value stored in Memory Slot 1 (ie the line number of the row we just added to appearance.2da) to the appearancenumber, appearance_s and appearance_l columns (since we only added one body size in appearance.2da). We change the value of the forpc column to be player selectable, and fill in the missing references to the Darkside portrait variations. (Lastly the row index of the new row is saved in Memory Slot 3, which is used for assigning that value to the PortraitId field in the creature blueprint above.)

 

* * *

 

It is important to remember the order in which things are done when storing values in the 2DAMEMORY tokens to be re-used later. The Patcher always merges the TLK files first, then modifies the 2DAs and modifies the GFF files last. Under each section of instructions things are then done in the order they are listed. I.e. files are modified in the order they are listed, and invidivual modifications for each file is done in the order they appear in the modifier lists.

 

* * *

 

This example just demonstrated CopyRow since this post is long enough as it is, but the other modifiers work in a fairly similar way. Check the comments in the changes.ini file, I've done my best to add comments to describe what options are available there. If you have any questions, please ask.

 

 

(I should probably make a GUI application that will create/edit the changes.ini file, but that will have to wait until I have the GFF editing component done.)

[RevanA4]

can i test too

[Darkkender]

You know stoffe that was going to be my complaint was the method in which a person has to use to add entries to a 2da file. I figured it out just fine and it will serve my purposes when I want to add a column to a 2da file.

 

On the note of adding columns to 2da files. Not all 2da files can accept a new column being added. Appearance.2da is one of them. Somewhere within the hardcoded elements of the game this 2da file has a defined number of columns. However there are many 2da's that seem to be fine with this. Such as if I recall a previous discussion with stoffe classpowergain.2da.

 

Just figured I would squish any newbie hopes with the adding columns features. I couldn't recall why in the past it was said that you can't add a new body type column to appearance.2da untill I tried it the hard way myself. Even if you add your columns to the end of the 2da file.

[stoffe]

Originally posted by Darkkender

You know stoffe that was going to be my complaint was the method in which a person has to use to add entries to a 2da file. I figured it out just fine and it will serve my purposes when I want to add a column to a 2da file.

 

Well, the TSLPatcher is primarily intended as an "installer" type of application, rather than a modding tool. (Even though things can be used for more than they are originally intended ).

 

It was made to make it a bit easier to merge different mods that modify the same 2da files, and as a way of using custom TLK entries in 2da files without having to distribute a whole modified dialog.tlk file. As such I figured that the INI file way of telling it what to do would suffice. It hopefully shouldn't be too hard to come up with some kind of GUI for it though, even if one would expect modders not to be afraid of text commands.

 

Originally posted by Darkkender

On the note of adding columns to 2da files. Not all 2da files can accept a new column being added. Appearance.2da is one of them. Somewhere within the hardcoded elements of the game this 2da file has a defined number of columns. However there are many 2da's that seem to be fine with this.

 

In the bioware spec for the 2da format they mention that some 2da files can be indexed by column number rather than the column label, and as such the ordering of columns is important. If a 2da is column number indexed the game code that uses it likely wouldn't read beyond the expected column count.

 

Don't know if this is the case with appearance.2da or if the number of available BodyVariations is just hardcoded in some other way though.

 

 

* * *

 

As an aside I just managed to put together a class that reads and writes GFF files. Hopefully it shouldn't take more than a few days to get it integrated with the TSLPatcher. (That will make it capable of updating for example the baseitem id in item blueprints etc to match newly inserted 2da rows.)

 

Still haven't heard a word from anyone else testing the TSLPatcher though. If this means that everyone is busy with more important things or that there are no bugs to report is open for interpretation, I suppose...

[envida]

Yeah sorry for not posting anything. But I have had very little time lately for modding

That said I have had the chance to test it a little but only adding custom entries to tlk file and so far I havent run into any problemes with that. I added some new entries and succesfully managed to use them in an custom display box (using DisplayMessageBox)

 

Great application and it shows much promise What would be great if it was possible to supply an installer with a mod that would automatically add any custom entires to the tlk file and 2da files....Come to think of it I guess thats possible with this program, you could just include a custom changes.ini and append.tlk file

[stoffe]

Originally posted by envida

Great application and it shows much promise What would be great if it was possible to supply an installer with a mod that would automatically add any custom entires to the tlk file and 2da files....Come to think of it I guess thats possible with this program, you could just include a custom changes.ini and append.tlk file

 

Yes, the original intention behind this application was to act as something of a distribution installer for my new force powers mod, since it uses custom names and descriptions for the force powers.

 

It has grown a bit in ability since then though since I figured others might find it useful too, so I added some more functions than I originally needed.

 

The idea was for those finding it useful to replace all files in the tslpatchdata folder with their own custom ones and put that folder along with the patcher app in the archive with their mod.

 

Originally posted by envida

That said I have had the chance to test it a little but only adding custom entries to tlk file and so far I havent run into any problemes with that. I added some new entries and succesfully managed to use them in an custom display box (using DisplayMessageBox)

 

Sounds good, I certainly don't mind if no one manages to find any bugs.

 

Speaking of scripts, I was thinking of adding some fairly limited support for patching NCS files as well.

 

It would not be as generic as the 2DA/TLK/GFF Patcher functionality is, since I'm hardly skilled enough to write a script compiler. I was thinking more about finding and replacing easily located values in the files, such as action function parameters (like DisplayMessageBox which you mentioned). It would just look for a specific set of functions that takes StrRefs or 2da-indexes as parameters and allow value substitution for recently inserted rows.

 

Does this sound like something that would be worth putting time into? I'm just at the thinking stage of that yet...

 

* * *

 

On another note, I just managed to add GFF patching functionality to the TSLPatcher. It can now take values (such as indexes of newly inserted 2DA lines) and assign them to a field label within a specified GFF file.

 

I've done some quick tests with inserting new appearance.2da and portraits.2da lines and have the patcher update the Appearance_Type and PortraitId fields for a bunch of creature blueprints, and it appeared to work fine. Though I'll have to do more testing before I'd be comfortable to say it fully works.

[stoffe]

I believe I have managed to iron out the most obvious bugs with the GFF patching now, so it should be safe enough for others to test.

 

I have updated the archive with this new version. It can be downloaded from the same URL as before. It also includes an updated version of TalkEd where I fixed a few bugs in that application that were reported to me.

 

I have added a new chapter to my novel in the comment lines of the changes.ini file that deals with GFF patching. So hopefully it shouldn't be too impossible to figure out how it works (unless you fall asleep halfway through reading it...).

 

Still, since the GFF bit is a fair bit more complex behind the scenes than the TLK and 2DA bit, I would be surprised if there isn't a bug or two that has managed to evade my notice. If you find anything that seems to work odd, please let me know.

[ChAiNz.2da]

Originally posted by stoffe -mkb-

and one oversight that prevented you from using StrRef tokens when assigning values to GFF fields.)

 

--snip--

 

Further, I am also about halfway done with a simple GUI application that will edit/generate a changes.ini file for the Patcher in a (hopefully) more user friendly way. It will probably be finished in a few days.

Well, that saved me a PM Someone beat me to it .. hehe

 

The GUI option sounds great stoffe! One of the things my simple mind was having problems with (in the beginning) was "what do I type here?" sort of stuff, but so far this tool is amazing, looking forward to every version you release

[stoffe]

I have uploaded a new version (same URL as before) that fixes two somewhat serious bugs that were reported, both related to using custom TLK entries and assigning StrRefs to 2DA cells or GFF fields.

 

(One crash bug if you added more than 3 String tokens (doh!), and one oversight that prevented you from using StrRef tokens when assigning values to GFF fields.)

 

I also fixed some odd behavior in TalkEd when listing recently added text strings, and removed the annoying feedback popup box after adding a new entry to the TLK file.

 

(EDIT: I also just fixed a new bug with the TLK tokens caused by a mistake when I fixed the previous problem. I have fixed this and re-uploaded a version (v0.8.9.0) that works correctly. If you downloaded the archive within the last hour or so (v0.8.8.0) you'll have to download it again. Sorry about that. Should probably change my test criteria a bit to avoid something like that in the future...)

 

* * *

 

Further, I am also about halfway done with a simple GUI application that will edit/generate a changes.ini file for the Patcher in a (hopefully) more user friendly way. It will probably be finished in a few days.