New JK2 Log parser is out

[Wintrmte]

I've completed the latest version of the log analyzer. You can grab the file by clicking http://jk2.gamersimpact.com

 

To see the sample report, click: http://jk2.gamersimpact.com/player-stats.html

Lots of new features and changes in this version.

 

From the CHANGES file:

 

CHANGES

 

v.0.9.8

Fixes:

 

o My pattern matching was a little bit off, therefore it was mixing weapons of like types.

Weapons like MOD_DISRUPTOR and MOD_DISRUPTOR_SNIPER were getting mixed in together.

-- Thanks to HellfireSK for pointing this out.

 

Additions:

 

o Script settings are no longer set within the script. Please see the logfile.conf to set all your settings with. There are new

config settings, so please view this file.

 

 

o Added 3 command line options to the script. They are:

-reinit : This will reinitialize (wipe) the players.dat file. This file is the historical player file.

-config <filename> : Path and Filename of your configuration file. The default is logparse.conf

-h : Display a help screen

 

o The script will now keep player statistics in a players.dat file.

-- Thanks MatrixCPA for the suggestion.

 

o Added a file that contains nicknames of players in which you do NOT want to keep statistics for. Names in the sample

player.filter file are:

Padawan

world

-- Thanks HellfireSK for the suggestion.

 

 

o Added individual player statistics. The script will create html pages for each player it's keeping stats for. The HTML for

the individual player stats can be customized through a template file. See the sample player-statfile.html

 

o The ratio's now include a nice bar image. There are 3 files included with this distribution. They are right.gif, left.gif and

main.gif. They need to be put somewhere inside your web accessible directory. Usually in /public_html/images . Note that the

directory location needs to match what you set in the logparse.conf file.

 

o Added the Suicide and Skill columns to the player statistics.

+ The suicide's are calculated by the player getting killed by <world> and if the player is killed by MOD_SUICIDE

+ The skill is calculated by # Kills - # Deaths - # Suicides = Skill

 

o You can now set the Servername, Administrator, and Admin Email values from within the logparse.conf file

 

o The nickname colorizer sub routine has been fixed. Thanks to Jim Walsh for sending the code in on this one.

 

TODO:

 

o Fix the WEAPONS_TO_DISPLAY setting, so that it actually displays weapons you specify

 

Thanks to everyone who sent their suggestions!

[Hell Raiser]

That is just soooooooooooooo friggin cool!

 

A thing I noticed: It doesn't do the color coding for stuff like ^d, and doesn't do the color coding at all for individual persons pages. Like I click on Shadow, and look at the people he fragged, their names aren't colored.

 

Also, it'd be nice if the weapons/what hurt wasn't listed as MOD_BLASTER or whatnot, just have a renaming sub routine in there to name it so it looks "normal" or something.

 

All in all, a very sweet sweet thing to have for a server. Good job.

 

/me gives it 2 thumbs and two big toes..... up!

[Wintrmte]

ACK!! LOL..

 

The not displaying the color'd names in the output for individual stats is fixed now. If you downloaded it previously, and want the colored names in the individual stats, redownload the file. Great catch!

 

Where can I find the control codes for like ^d and ^m? I found the color codes, but couldn't find the letter ones..

 

Also, you can change the weapon mappings from within the script, with full instructions on how to do it.

[Hell Raiser]

Custimizable is good.

 

Well, as far as I can tell, letters run through the colors in sequence like the numbers. A is 1, B is 2, ect ect, and it just repeats over and over.

 

Definatly gonna download this and run it on my server once I get it up and working.

[pcmacman]

I have a quick question for you: I downloaded the first version of the stats that you had in this thread, and I'm having lots of trouble getting the colored bars to work. Inside the config file it says to give the image directory inside the output directory i have it like this:

 

 

###############################################

# This is your web files directory #

###############################################

OUTPUTDIR /home/pcmacman/public_html/jedi

 

#and

 

##########################################################################################

# The images directory is the location INSIDE your outputdir setting that the script can #

# find the images files for the web output. #

##########################################################################################

IMAGESDIR /images

 

 

So I assume that the perl script will look in /home/pcmacman/public_html/jedi/images for the *.gif files. But if you look at the stat site you can see that its looking in http://stompbox.dhs.org:81/images for the gif files.

 

I was going to go into logparse.pl and take out the error protection on the images directory, but I thought i'd check here first. And it would be much easier if there was a better fix rather than me ediditng the file each time.

 

Am I missing something?

[Wintrmte]

Originally posted by pcmacman

I have a quick question for you: I downloaded the first version of the stats that you had in this thread, and I'm having lots of trouble getting the colored bars to work. Inside the config file it says to give the image directory inside the output directory i have it like this:

 

 

 

 

So I assume that the perl script will look in /home/pcmacman/public_html/jedi/images for the *.gif files. But if you look at the stat site you can see that its looking in http://stompbox.dhs.org:81/images for the gif files.

 

I was going to go into logparse.pl and take out the error protection on the images directory, but I thought i'd check here first. And it would be much easier if there was a better fix rather than me ediditng the file each time.

 

Am I missing something?

 

Edit:

 

Ok, I found a problem in my last post. I've removed the IMAGESDIR checking from the script. If you want to remove this, you can, otherwise go ahead and download the new script.

 

If you specify (without the imagesdir checking) /jedi/images then it will work fine.

 

Sorry about that...

 

Edit:

 

You might also want to check the STYLESHEET value. I noticed none of the stylesheet settings were coming across on your site. This too is relative to your OUTPUTDIR. If you placed the stats.css file inside your /jedi directory, set the STYLESHEET to be just stats.css

[MatrixCPA]

Ok, I fixed it so it doesn't count suicides as kills again. It was kinda funny to see people who's "#1 target" was themselves! LOL! Anyway, if you'd like to review what I changed, just contact me via ICQ (74850239), AIM (MatrixCPA), or MSN Messenger (matrixcpa@msn.com) and I'd be happy to share.

[geronimo27]

Is there anything with this script to prevent it from working on win2k?

[MatrixCPA]

It's a perl script, so as long as you have perl installed you should be fine. I remember seeing that 5.005 was required, but there really isn't much reason not to use 5.6.1

[geronimo27]

I was finally able to get this to work.

 

Now, my last question, is how do I set it up so that it automatically runs the parser every so often without me having to manually run it?

 

Thanks in advance!

[Leo]

It doesn't accept the path D:/Program Files ? Changing it at D:/Programfiles How I fix that ?

[DOM SlyFox]

First of all thanks for the hard work! Now for my issue. I got the first verison running just fine. I can't seem to get the second running tho. After I exec the logparse.pl file, it says " Cannot find blah/blah/blah/jedi.log file... I know it is there. It is in the same place as it was for your first version and it found it and worked. Any ideas? I even went as far as editing the .pl file to show the location of the $LOGFILE, the same way you have it find the .conf file. Am I missing something. Should the path in the .conf file be the same as I had it in the original .pl file?

 

Thanks for any help...

[Leo]

Found it. Must use the DOS format, here is mine :

 

D:\PROGRA~1\LUCASA~1\STARWA~1\GAMEDATA\base\games.log

[DOM SlyFox]

Thanks Leo, will give that a shot when I get home!

 

 

err.... but one thing, I move the logfile out to my unix web server, so I pretty much think I have the correct path, it worked in the first version. Was just wondering if something changed in the way you show the path with the addition of the new .conf file.

[Wintrmte]

Originally posted by DOM SlyFox

Thanks Leo, will give that a shot when I get home!

 

 

err.... but one thing, I move the logfile out to my unix web server, so I pretty much think I have the correct path, it worked in the first version. Was just wondering if something changed in the way you show the path with the addition of the new .conf file.

 

The only thing that changed in the way I read the paths, was that I was stripping out white spaces from the path (which, after someone pointed it out to me, was a bad thing).

 

So, I changed the script yet again so that it won't strip out white spaces. The example of C:\Program Files\something not working was correct. Because I was stripping out white spaces, it made the path look like C:\ProgramFiles

 

Now, as far as it not being able to find your jedi.log file, make sure you put in the FULL Path. If you don't specify a leading / in front of your path, then the script starts looking in the directory it's being run from, and moves from there. So, if you say your LOGFILE path is:

 

blah/blah/jedi.log

 

Then, the script is going to look in the same directory as it's self, and want to find the logfile in directory blah/blah

 

Best way to do it, is to fully qualify the path to your logfile (i.e.:

 

/home/userid/logfile/jedi.log

[DOM SlyFox]

Ok, thanks for the info, I do have the full path set, i must be doing something else wrong i will check it out more when I get off work.

 

Thanks again!

[Leo]

Very nice job Wintrmte, I am thinking of 2 features that would be really nice :

 

Ability for the perl script to auto-exec it's self at a given time.

Ability to auto-upload the html files through ftp.

[rivmin]

Regarding version 1.0.4.1

 

-config <filename> is not supported...and it's quite bad, since I run several servers....hopefully this will be fixed soon , so I can make stat-files from the other servers running [dont want to install 1 parser pr server, though it would work ]

 

-path to template files has to be modify'd inside the script, this should be in the config-file.

 

-stylesheet-location is not pointing to the correct dir.

 

-images-location is not pointing to the correct dir.

 

---

 

My output is here:

http://uv.orangenet.dk/stat2/jk1/player-stats.html

 

When -config <filename> is working, I will parse the other servers.

 

--

 

Keep up the good work

[geronimo27]

Btw, yes, thanks for putting your time and efforts into this - the stats are very nice.

 

One issue I ran into late last night, which I need to look at more carefully when I get back home from work, has to do with the templates that are used. I changed the template names from "main-stats.html" to "stats.html" and from "player-stats(is that right?).html" to "players.html", and made the same necessary changes to the temp names in the script.

 

But, it seems when I first run the script, the main stats are written to players.html, and then when I look at individual players' stats pages, they all show the main stats as well.

 

So, it looks like for some reason players.html is getting used to write the new stats, and then when it comes time to create the individual player stats, it uses the players.html template, which now contains all of the main stats, hence the player temps show the main stats.

 

I hope that made sense

[HellfireSK]

Well done, the .conf works quite nicely.

 

The comments on the ImageDir directive are a teeny bit confusing, it might be more easily explained as simply a virtual directory relative to your web root. Trial and error show you what's what, but commentary is always nice.

 

Can you elaborate a bit on "the procedure" for rotating logs in and out? For example, a moderate use CTF server generated a 10mb log file inside of a few days. Obviously since histories are preserved now I want to rotate them out to speed up the parsing as well as keep things tidy.

 

I parsed the 10mb log a while ago, rotated it out and then cron parsed the brand new 10k log, stating that instead of 320k lines it had parsed 200. The stats however, incremented as expected.

 

-h

[MatrixCPA]

Originally posted by Leo

Very nice job Wintrmte, I am thinking of 2 features that would be really nice :

 

Ability for the perl script to auto-exec it's self at a given time.

 

Well, on 'nix that's the beauty of cron. Although, and this is for all 'nix admins who use cron to run the stats, you'll need to edit the script and customize the location of the conf file. It's just set as the current dir and thus when you run it from cron, it can't find it.

[geronimo27]

Also, what exactly is meant by "rotating out the log"? Do you just mean shut down the server, nuke the log file, then start server again?

[Wintrmte]

Originally posted by rivmin

Regarding version 1.0.4.1

 

-config <filename> is not supported...and it's quite bad, since I run several servers....hopefully this will be fixed soon , so I can make stat-files from the other servers running [dont want to install 1 parser pr server, though it would work ]

 

-path to template files has to be modify'd inside the script, this should be in the config-file.

 

-stylesheet-location is not pointing to the correct dir.

 

-images-location is not pointing to the correct dir.

 

---

 

My output is here:

http://uv.orangenet.dk/stat2/jk1/player-stats.html

 

When -config <filename> is working, I will parse the other servers.

 

--

 

Keep up the good work

 

Ok. -config was wrong in the CHANGES file. It should have read -c .. If you use -c <config file> then it works. I made the change to the CHANGES file, and also the perldoc information.

 

As far as the images directory goes, if you are using version 1.0.4.1 of the parser, that should be fixed.

 

Remember, both stylesheets and imagesdir are relative to the base of your web directory.

 

The best way to deal with the stylesheet and imagesdir is to fully qualify them within your root web directory.

 

i.e.

 

if your images directory is inside a directory called jedistats (and jedistats is off the ROOT of your web directory ), put:

 

IMAGEDIR /jedistats/images

 

likewise, if your stylesheet is in /jedistats/stylesheets

 

STYLESHEET /jedistats/stylesheets/stats.css

 

Modifying the template stuff will go into the .conf file in the next version.

[Wintrmte]

Originally posted by HellfireSK

Well done, the .conf works quite nicely.

 

The comments on the ImageDir directive are a teeny bit confusing, it might be more easily explained as simply a virtual directory relative to your web root. Trial and error show you what's what, but commentary is always nice.

 

Can you elaborate a bit on "the procedure" for rotating logs in and out? For example, a moderate use CTF server generated a 10mb log file inside of a few days. Obviously since histories are preserved now I want to rotate them out to speed up the parsing as well as keep things tidy.

 

I parsed the 10mb log a while ago, rotated it out and then cron parsed the brand new 10k log, stating that instead of 320k lines it had parsed 200. The stats however, incremented as expected.

 

-h

 

By rotating the logs, if you move the current jedi.log file to say, jedi.2002-04-08.log, and then the new jedi.log gets created. I'm not suggesting doing a nightly rotation or anything like that, but,

if you do start your logfile over, then you need to run the logparse.pl again so that it can update it's counter correctly.

 

In regards to it only showing that it parsed 200 lines of log files after you rotated it, that would be correct (base on whatever your server output during the time since it was started). I don't keep historical data on log lines that were read.

[rivmin]

Originally posted by Wintrmte

 

By rotating the logs, if you move the current jedi.log file to say, jedi.2002-04-08.log, and then the new jedi.log gets created. I'm not suggesting doing a nightly rotation or anything like that,

 

Just to make it clear [to me], it will be a good idea to rename the current logfile, to minimize ressource-usage ?

 

Every time I run the parser...does it allways run all the way through the logfile, from line 1 to current line...or does it make a 'bookmark', where it will continue from ?

 

What do u suggest as a logical/smart/intelligent number-of-lines to rename the current log-file ?

 

I am now on line 110.000 according to the LogParser...

 

--

 

I run the LogParser every hour with this simple script, that I screen:

 

cd /usr/home/uv/jk2stat22

while true

do

./logparse.pl

sleep 3600

done

 

--

 

I run my servers with this script, that I also screen:

 

while true

do

cd /usr/home/uv/jk2

date>>jk1serverrestart.log

./jk2ded +set net_port 28070 +sets gamestartup \"`date +"%D %T"`\" +exec server.cfg

sleep 10

done

 

It will restart the server if crashed, and make a timestamp in a restart-log-file, so I can see how stable it is.

 

OS is FreeBSD4.5.

 

Stat-output: http://uv.orangenet.dk