Compiling in Linux

[eyen]

I've been trying to compile my server mod in Linux, I patch it and then run make and it creates a jampgamei386.so (with a lot of warnings in the process of doing that). I put this in my base folder and started up a server and go in to the game. Now the problem is that in the game I can't see any players, objects or movers until I'm standing close enough to touch them with a saber at the very least. I've checked that the original .so worked fine and did not cause this problem. I've also compiled the "universal SDK" which was already patched and get the same (or at least most of the same, too many to count) warnings and when the server is started up it does indeed have the same problem. So the problem I assume must be in making the .so but I can't figure out what's wrong, all of the warnings are about implicit declaration of memset/memcpy/strcpy and things like that and I'm not sure if that's what's causing the problem.

 

I'd really appreciate it if someone was able to help me out with this one.

[ensiform]

This is generally an issue with gcc4. Not sure why, some people get it and others do not. BTW, with gcc4 you should make sure that stack smashing protection is not being run.

 

If you are using Debian or Ubuntu (or its siblings, Kubuntu, Xubuntu, etc)... then it will be enabled by default in the kernel somewhere. You can disable it in your makefile though using:

 

-fno-stack-protector

 

That flag is pretty useless for gaming as it just errors out during run-time. Most people want to debug their stuff at compile time. ;x

 

I use SCons for my building now anyway, which uses gcc/g++ but its more efficient at being cross-platform and automake and what not just don't cut it for me. Plus python rocks for building configurations... I borrowed mine from the ETPub guys.

[ensiform]

In case you are interested...

 

I haven't finished with getting the mac stuff since I don't actually build for MAC myself and I haven't toyed with the pk3 part yet either...

 

SConstruct:

 

# -*- mode: python -*-

# Jedi Academy build script

# TTimo <ttimo@idsoftware.com>
# Ensiform <ensiform[at]gmail[dot]com>

# http://scons.sourceforge.net



import sys, os, time, commands, re, pickle, StringIO, popen2, commands, pdb, zipfile, string, tempfile

import SCons



import scons_utils



conf_filename='site.conf'

# choose configuration variables which should be saved between runs

# ( we handle all those as strings )

serialized=[ 'CC', 'CXX', 'JOBS', 'BUILD', 'BUILD_ROOT', 'TARGET_GAME', 'TARGET_CGAME', 'TARGET_UI' ]



# help -------------------------------------------



Help("""

Usage: scons [OPTIONS] [TARGET] [CONFIG]



[OPTIONS] and [TARGET] are covered in command line options, use scons -H



[CONFIG]: KEY="VALUE" [...]

a number of configuration options saved between runs in the """ + conf_filename + """ file

erase """ + conf_filename + """ to start with default settings again



CC (default gcc)

CXX (default g++)

Specify C and C++ compilers (defaults gcc and g++)

ex: CC="gcc-3.3"

You can use ccache and distcc, for instance:

CC="ccache distcc gcc" CXX="ccache distcc g++"



JOBS (default 1)

Parallel build



BUILD (default debug)

Use debug-all/debug/release to select build settings

ex: BUILD="release"

debug-all: no optimisations, debugging symbols

debug: -O -g

release: all optimisations, including CPU target etc.



DEDICATED (default 2)

Control regular / dedicated type of build:

0 - client

1 - dedicated server

2 - both



TARGET_GAME (default 1)

Build game module



BUILD_ROOT (default 'build')

change the build root directory



NOCONF (default 0, not saved)

ignore site configuration and use defaults + command line only



COPYBINS (default 0, not saved)

copy the binaries in a ready-to-release format



BUILDMPBIN (default 0, not saved)

build mp_bin.pk3 using bin/ directories and game binaries for all platforms



BUILDBUNDLE (default 0, not saved)

create mac bundle files

"""

)



# end help ---------------------------------------



# sanity -----------------------------------------



EnsureSConsVersion( 0, 96 )



# end sanity -------------------------------------



# system detection -------------------------------



# CPU type

cpu = commands.getoutput('uname -m')

dll_cpu = '???' # grmbl, alternative naming for .so

exp = re.compile('.*i?86.*')

if exp.match(cpu):

cpu = 'x86'

dll_cpu = 'i386'

else:

cpu = commands.getoutput('uname -p')

if ( cpu == 'powerpc' ):

	cpu = 'ppc'

	dll_cpu = cpu

else:

	cpu = 'cpu'

	dll_cpu = cpu

OS = commands.getoutput( 'uname -s' )



print 'cpu: ' + cpu



# end system detection ---------------------------



# default settings -------------------------------



CC = 'gcc'

CXX = 'g++'

JOBS = '1'

BUILD = 'debug'

DEDICATED = '2'

TARGET_GAME = '1'

TARGET_CGAME = '1'

TARGET_UI = '0'

BUILD_ROOT = 'build'

NOCONF = '0'

COPYBINS = '0'

BUILDMPBIN = '0'

BUILDBUNDLE = '0'



# end default settings ---------------------------



# arch detection ---------------------------------



def gcc_major():

major = os.popen( CC + ' -dumpversion' ).read().strip()

print 'dumpversion: %s' % major

major = re.sub('^([^.]+)\\..*$', '\\1', major)

print 'gcc major: %s' % major

return major



gcc3 = (gcc_major() != '2')



def gcc_is_mingw():

mingw = os.popen( CC + ' -dumpmachine' ).read()

return re.search('mingw', mingw) != None



if gcc_is_mingw():

g_os = 'win32'

elif cpu == 'ppc':

g_os = 'Darwin'

else:

g_os = 'Linux'

print 'os: %s' % g_os



# end arch detection -----------------------------



# site settings ----------------------------------



if ( not ARGUMENTS.has_key( 'NOCONF' ) or ARGUMENTS['NOCONF'] != '1' ):

site_dict = {}

if (os.path.exists(conf_filename)):

	site_file = open(conf_filename, 'r')

	p = pickle.Unpickler(site_file)

	site_dict = p.load()

	print 'Loading build configuration from ' + conf_filename + ':'

	for k, v in site_dict.items():

		exec_cmd = k + '=\'' + v + '\''

		print '  ' + exec_cmd

		exec(exec_cmd)

else:

print 'Site settings ignored'



# end site settings ------------------------------



# command line settings --------------------------



for k in ARGUMENTS.keys():

exec_cmd = k + '=\'' + ARGUMENTS[k] + '\''

print 'Command line: ' + exec_cmd

exec( exec_cmd )



# end command line settings ----------------------



# save site configuration ----------------------



if ( not ARGUMENTS.has_key( 'NOCONF' ) or ARGUMENTS['NOCONF'] != '1' ):

for k in serialized:

	exec_cmd = 'site_dict[\'' + k + '\'] = ' + k

	exec(exec_cmd)



site_file = open(conf_filename, 'w')

p = pickle.Pickler(site_file)

p.dump(site_dict)

site_file.close()



# end save site configuration ------------------



# arch detection ---------------------------------



def gcc_major():

major = os.popen( CC + ' -dumpversion' ).read().strip()

major = re.sub('^([^.]+)\\..*$', '\\1', major)

print 'gcc major: %s' % major

return major



gcc3 = (gcc_major() != '2')



def gcc_is_mingw():

mingw = os.popen( CC + ' -dumpmachine' ).read()

return re.search('mingw', mingw) != None



win32_build = gcc_is_mingw()



# end arch detection -----------------------------



# general configuration, target selection --------



g_build = BUILD_ROOT + '/' + BUILD



SConsignFile( 'scons.signatures' )



SetOption('num_jobs', JOBS)



if ( OS == 'Linux' ):

LINK = CC

else:

LINK = CXX



# common flags

# BASE + GAME + OPT for game



BASECPPFLAGS = [ ]

CORECPPPATH = [ ]

CORELIBPATH = [ ]

CORECPPFLAGS = [ ]

GAMECPPFLAGS = [ ]

BASELINKFLAGS = [ ]

CORELINKFLAGS = [ ]



# for release build, further optimisations that may not work on all files

OPTCPPFLAGS = [ ]



if ( OS == 'Darwin' ):

# taken from xcode's default project settings

BASECPPFLAGS += [ '-D__MACOS__', '-Wno-long-double', '-arch', 'ppc', '-fasm-blocks', '-fpascal-strings', '-faltivec', '-mcpu=G4', '-mtune=G4', '-Wno-unknown-pragmas' ]



BASECPPFLAGS.append( '-pipe' )

# warn all

BASECPPFLAGS.append( '-Wall' )

# don't wrap gcc messages

BASECPPFLAGS.append( '-fmessage-length=0' )



if ( BUILD == 'debug-all' ):

BASECPPFLAGS.append( '-g' )

BASECPPFLAGS.append( '-D_DEBUG' )
BASECPPFLAGS.append( '-fno-stack-protector' )

elif ( BUILD == 'debug' ):

BASECPPFLAGS.append( '-g' )

BASECPPFLAGS.append( '-O1' )

BASECPPFLAGS.append( '-D_DEBUG' )
BASECPPFLAGS.append( '-fno-stack-protector' )

elif ( BUILD == 'release' ):

BASECPPFLAGS.append( '-DNDEBUG' )
BASECPPFLAGS.append( '-DFINAL_BUILD' )
BASECPPFLAGS.append( '-fno-stack-protector' )

if ( OS == 'Linux' ):

	# -fomit-frame-pointer: gcc manual indicates -O sets this implicitely,

	# only if that doesn't affect debugging

	# on Linux, this affects backtrace capability, so I'm assuming this is needed

	# -finline-functions: implicit at -O3

	# -fschedule-insns2: implicit at -O3

	# -funroll-loops ?

	# -mfpmath=sse -msse ?
	OPTCPPFLAGS = [ '-march=i686', '-mmmx', '-fexpensive-optimizations', '-O3', '-static' ]

elif ( OS == 'Darwin' ):

	OPTCPPFLAGS = [ '-O3', '-falign-functions=16', '-falign-loops=16', '-finline' ]

else:

print 'Unknown build configuration ' + BUILD

sys.exit(0)



# create the build environments

g_base_env = Environment( ENV = os.environ, CC = CC, CXX = CXX, LINK = LINK, CPPFLAGS = BASECPPFLAGS, LINKFLAGS = BASELINKFLAGS, CPPPATH = CORECPPPATH, LIBPATH = CORELIBPATH )

scons_utils.SetupUtils( g_base_env )



g_env = g_base_env.Copy()



g_env['CPPFLAGS'] += OPTCPPFLAGS

g_env['CPPFLAGS'] += CORECPPFLAGS

g_env['LINKFLAGS'] += CORELINKFLAGS



if ( OS == 'Linux' ):

g_env.Append(LINKFLAGS = '-rdynamic')

g_env.Append(LINKFLAGS = '-m32')

g_env.Append(CPPFLAGS = '-m32')



if ( OS == 'Darwin' ):

# configure for dynamic bundle

g_env['SHLINKFLAGS'] = '$LINKFLAGS -bundle -flat_namespace -undefined suppress'

g_env['SHLIBSUFFIX'] = '.so'



# maintain this dangerous optimization off at all times

g_env.Append( CPPFLAGS = '-fno-strict-aliasing' )



if ( int(JOBS) > 1 ):

print 'Using buffered process output'

scons_utils.SetupBufferedOutput( g_env )



# mark the globals



GLOBALS = 'g_env OS g_os BUILD gcc3 cpu'



# end general configuration ----------------------



# win32 cross compilation ----------------------



if g_os == 'win32' and os.name != 'nt':

# mingw doesn't define the cpu type, but our only target is x86

g_env.Append(CPPDEFINES = '_M_IX86=400')

g_env.Append(LINKFLAGS = '-static-libgcc')

g_env.Append(LIBS = 'ws2_32')

# scons doesn't support cross-compiling, so set these up manually

g_env['STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME'] = 1

g_env['WIN32DEFSUFFIX']	= '.def'

g_env['PROGSUFFIX']	= '.exe'

g_env['SHLIBSUFFIX']	= '.dll'

g_env['SHCCFLAGS']	= '$CCFLAGS'



# end win32 cross compilation ------------------



# targets ----------------------------------------



toplevel_targets = []



if ( TARGET_GAME == '1' ):

Export( 'GLOBALS ' + GLOBALS )

BuildDir( g_build + '/game', '.', duplicate = 0 )

game = SConscript( g_build + '/game/SConscript.game' )

if ( g_os == 'win32' ):

	toplevel_targets.append( InstallAs( '#jampgamex86.dll', game ) )

else:

	toplevel_targets.append( InstallAs( '#jampgamei386.so', game ) )



if ( TARGET_CGAME == '1' ):

Export( 'GLOBALS ' + GLOBALS )

BuildDir( g_build + '/cgame', '.', duplicate = 0 )

cgame = SConscript( g_build + '/cgame/SConscript.cgame' )

if ( g_os == 'win32' ):

	toplevel_targets.append( InstallAs( '#cgamex86.dll', cgame ) )

else:

	toplevel_targets.append( InstallAs( '#cgame.i386.so', cgame ) )



if ( TARGET_UI == '1' ):

Export( 'GLOBALS ' + GLOBALS )

BuildDir( g_build + '/ui', '.', duplicate = 0 )

ui = SConscript( g_build + '/ui/SConscript.ui' )

if ( g_os == 'win32' ):

	toplevel_targets.append( InstallAs( '#ui_mp_x86.dll', ui ) )

else:

	toplevel_targets.append( InstallAs( '#ui.i386.so', ui ) )



class CopyBins(scons_utils.idSetupBase):

def copy_bins( self, target, source, env ):

	for i in source:

		j = os.path.normpath( os.path.join( os.path.dirname( i.abspath ), '../bin', os.path.basename( i.abspath ) ) )

		self.SimpleCommand( 'cp ' + i.abspath + ' ' + j )

		if ( OS == 'Linux' ):

			self.SimpleCommand( 'strip ' + j )

		else:

			# see strip and otool man pages on mac

			self.SimpleCommand( 'strip -ur ' + j )



copybins_target = []

if ( COPYBINS != '0' ):

copy = CopyBins()

copybins_target.append( Command( 'copybins', toplevel_targets, Action( copy.copy_bins ) ) )



class MpBin(scons_utils.idSetupBase):

def mp_bin( self, target, source, env ):

	temp_dir = tempfile.mkdtemp( prefix = 'mp_bin' )

	self.SimpleCommand( 'cp ../bin/ui* ' + temp_dir )

	self.SimpleCommand( 'cp ../bin/cgame* ' + temp_dir )

	# zip the mac bundles

	mac_bundle_dir = tempfile.mkdtemp( prefix = 'mp_mac' )

	self.SimpleCommand( 'cp -R "../bin/Wolfenstein ET.app/Contents/Resources/ui_mac.bundle" ' + mac_bundle_dir )

	self.SimpleCommand( 'cp -R "../bin/Wolfenstein ET.app/Contents/Resources/cgame_mac.bundle" ' + mac_bundle_dir )

	self.SimpleCommand( 'find %s -name \.svn | xargs rm -rf' % mac_bundle_dir )

	self.SimpleCommand( 'cd %s ; zip -r -D %s/ui_mac.zip ui_mac.bundle ; mv %s/ui_mac.zip %s/ui_mac' % ( mac_bundle_dir, temp_dir, temp_dir, temp_dir ) )

	self.SimpleCommand( 'cd %s ; zip -r -D %s/cgame_mac.zip cgame_mac.bundle ; mv %s/cgame_mac.zip %s/cgame_mac' % ( mac_bundle_dir, temp_dir, temp_dir, temp_dir ) )

	mp_bin_path = os.path.abspath( os.path.join ( os.getcwd(), '../etmain/mp_bin.pk3' ) )

	self.SimpleCommand( 'cd %s ; zip -r -D %s *' % ( temp_dir, mp_bin_path ) )



if ( BUILDMPBIN != '0' ):

mp_bin = MpBin()

mpbin_target = Command( 'mp_bin', toplevel_targets + copybins_target, Action( mp_bin.mp_bin ) )



class BuildBundle(scons_utils.idSetupBase):

def make_bundle( self, target, source, env ):

	for i in source:

		self.SimpleCommand( './makebundle.sh %s' % i )



if ( BUILDBUNDLE != '0' ):

build_bundle = BuildBundle()

bundle_target = Command( 'build_bundle', toplevel_targets, Action( build_bundle.make_bundle ) )



# end targets ------------------------------------

 

SConscript.game:

 

# -*- mode: python -*-

# Jedi Academy build script

# TTimo <ttimo@idsoftware.com>
# Ensiform <ensiform[at]gmail[dot]com>

# http://scons.sourceforge.net



import sys, os

import scons_utils



Import( 'GLOBALS' )

Import( GLOBALS )



botai_string = """

ai_main.c

ai_tab.c

ai_util.c

ai_wpnav.c

"""



botai_list = scons_utils.BuildList( 'game', botai_string )



jampgame_string = """

AnimalNPC.c

bg_g2_utils.c

bg_killtracker.c

bg_langscript.c

bg_misc.c
bg_panimate.c

bg_pmove.c
bg_saber.c
bg_saberLoad.c
bg_saga.c

bg_slidemove.c
bg_vehicleLoad.c
bg_weapons.c
FighterNPC.c

g_active.c
g_admin.c
g_antiwarp.c
g_bot.c
g_censor.c
g_client.c

g_cmds.c

g_combat.c
g_crash.c
g_exphysics.c
g_fakebrush.c
g_holocron.c
g_ICARUScb.c

g_items.c
g_log.c

g_main.c

g_mem.c

g_misc.c

g_missile.c

g_mover.c
g_nav.c
g_navnew.c
g_object.c
g_saga.c

g_session.c
g_settings.c
g_shrubbot.c

g_spawn.c
g_strap.c

g_svcmds.c

g_syscalls.c

g_target.c

g_team.c
g_timer.c

g_trigger.c
g_turret.c
g_turret_G2.c
g_unlagged.c

g_utils.c
g_vehicles.c
g_vehicleTurret.c

g_vote.c
g_weapcfg.c

g_weapon.c
NPC.c
NPC_AI_Atst.c
NPC_AI_Default.c
NPC_AI_Droid.c
NPC_AI_GalakMech.c
NPC_AI_Grenadier.c
NPC_AI_Howler.c
NPC_AI_ImperialProbe.c
NPC_AI_Interrogator.c
NPC_AI_Jedi.c
NPC_AI_Mark1.c
NPC_AI_Mark2.c
NPC_AI_MineMonster.c
NPC_AI_Rancor.c
NPC_AI_Remote.c
NPC_AI_Seeker.c
NPC_AI_Sentry.c
NPC_AI_Sniper.c
NPC_AI_Stormtrooper.c
NPC_AI_Utils.c
NPC_AI_Wampa.c
NPC_behavior.c
NPC_combat.c
NPC_goal.c
NPC_misc.c
NPC_move.c
NPC_reactions.c
NPC_senses.c
NPC_sounds.c
NPC_spawn.c
NPC_stats.c
NPC_utils.c
q_math.c
q_shared.c
SpeederNPC.c
tri_coll_test.c
w_force.c
w_saber.c
WalkerNPC.c

"""



jampgame_list = scons_utils.BuildList( 'game', jampgame_string )



local_env = g_env.Copy()

local_env.Append( CPPDEFINES = [ 'SCONS_BUILD' ] ) # Tell the code that we are building from SCONS

local_env.Append( CPPDEFINES = [ 'NO_OMNIBOT_HEADER' ] ) # Don't include the Omni-Bot headers for Linux just yet.



local_env.Append( CPPDEFINES = [ 'QAGAME' ] ) # Server



local_env.Append( CPPDEFINES = [ '_JK2' ] ) # Raven Define for "MP"

local_env.Append( CPPDEFINES = [ '__linux__' ] ) # enforce the linux define because I'm not sure that gcc does this already, and our code relies on this preprocessor define



local_env['LINK'] = local_env['CXX']



source_list = botai_list

source_list += jampgame_list



#ret = local_env.Program( target = 'et', source = source_list )

ret = local_env.SharedLibrary( target = 'jampgame', source = source_list)

Return( 'ret' )

[eyen]

Thanks for the reply ensiform, and it does seem very possible that stack smashing protection could be the problem as you specifically mentioned Debian which is what I'm using and I installed gcc-4.1 off the installation DVD. However I wasn't able to fix the problem by adding -fno-stack-protector to the CFLAGS part of the makefile (I assume that's where it goes, I haven't used Linux that extensively). But there's no way I can tell if that is actually disabling SSP as far as I can tell because it seems to accept anything I put in that section without telling me it doesn't exist (I tried a few things which I would expect would not exist).

 

I suppose I could try installing an older version of gcc if you think that might fix the problem, but it'd be nice to find a simple fix to it.. there's a high chance I've just not added the flag correctly.

[ensiform]

Well for compiling a release i use the following flags:

 

-pipe

-Wall (warn all)

-fmessage-length=0 ( don't wrap gcc msgs)

-march=i686

-mmmx ( I modified the math portions in q_shared and the q_math stuff to allow for mmx/sse/sse2 optimizations when compiling the server, cgame, and ui)

-fexpensive-optimizations

I rotate between using -O2 and -O3

-static (not required for you, it just puts glibc as static library instead of shared lib loading)

-fno-strict-aliasing

-rdynamic

-m32 (Link flag and cpp flag)

[eyen]

I installed gcc-3.3 and compiled with that and it worked perfectly first time, but I'll have another go at getting it to work in 4.1 sometime tomorrow. Also I'll try those flags out, is there much to be gained from optimizing JA?

[dumbledore]

you did remember to make clean after you added the new cflag right?