scons: Stop generating inc.d in the isa parser.

Generating dependency/build product information in the isa parser breaks scons
idea of how a build is supposed to work. Arm twisting it into working forced
a lot of false dependencies which slowed down the build.

Change-Id: Iadee8c930fd7c80136d200d69870df7672a6b3ca
Reviewed-on: https://gem5-review.googlesource.com/5081
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>

arch: teach ISA parser how to split code across files

This patch encompasses several interrelated and interdependent changes
to the ISA generation step. The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.

The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units. I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.

Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner. It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type. Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.

Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths. In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.

Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.

arch: Make the ISA class inherit from SimObject

The ISA class on stores the contents of ID registers on many
architectures. In order to make reset values of such registers
configurable, we make the class inherit from SimObject, which allows
us to use the normal generated parameter headers.

This patch introduces a Python helper method, BaseCPU.createThreads(),
which creates a set of ISAs for each of the threads in an SMT
system. Although it is currently only needed when creating
multi-threaded CPUs, it should always be called before instantiating
the system as this is an obvious place to configure ID registers
identifying a thread/CPU.

ISA: Back-out NoopMachInst as a StaticInstPtr change.

ISA: Turn the ExtMachInst NoopMachinst into the StaticInstPtr NoopStaticInst.

This eliminates a use of the ExtMachInst type outside of the ISAs.

ISA: Make the decode function part of the ISA's decoder.

SE/FS: Remove the last uses of FULL_SYSTEM from SPARC.

SPARC: Build vtophys into SE mode.

[mq]: sefssparcregfile.patch

SE/FS: Build the Interrupt objects in SE mode.

scons: rename TraceFlags to DebugFlags

SPARC: Get rid of the copy/pasted StackTrace stolen from Alpha.

CPU: Separate out native trace into ISA (in)dependent code and SimObjects.

SPARC: Fold the MiscRegFile all the way into the ISA object.

Registers: Add a registers.hh file as an ISA switched header.
This file is for register indices, Num* constants, and register types.
copyRegs and copyMiscRegs were moved to utility.hh and utility.cc.

Registers: Eliminate the ISA defined integer register file.

Registers: Eliminate the ISA defined floating point register file.

Registers: Add an ISA object which replaces the MiscRegFile.
This object encapsulates (or will eventually) the identity and characteristics
of the ISA in the CPU.

SPARC: Add a traceflag for register windows.

tracing: Add help strings for some of the trace flags

SCons: centralize the Dir() workaround for newer versions of scons.
Scons bug id: 2006 M5 Bug id: 308

Turn Interrupts objects into SimObjects. Also, move local APIC state into x86's Interrupts object.

SCons: Fixing SCons bug 2006 issues for non-alpha ISAs

Traceflags: Add SCons function to created a traceflag instead of having one file with them all.

Address Translation: Make SE mode use an actual TLB/MMU for translation like FS.

Arguments: Get rid of duplicate code for the Arguments class in each architecture.
Move the argument files to src/sim and add a utility.cc file with a function
getArguments() that returns the given argument in the architecture specific fashion.
getArguments() was getArg() is the architecture specific Argument class and has had
all magic numbers replaced with meaningful constants. Also add a function to the
Argument class for testing if an argument is NULL.

Move SimObject python files alongside the C++ and fix
the SConscript files so that only the objects that are
actually available in a given build are compiled in.
Remove a bunch of files that aren't used anymore.

Rework the way SCons recurses into subdirectories, making it
automatic. The point is that now a subdirectory can be added
to the build process just by creating a SConscript file in it.
The process has two passes. On the first pass, all subdirs
of the root of the tree are searched for SConsopts files.
These files contain any command line options that ought to be
added for a particular subdirectory. On the second pass,
all subdirs of the src directory are searched for SConscript
files. These files describe how to build any given subdirectory.
I have added a Source() function. Any file (relative to the
directory in which the SConscript resides) passed to that
function is added to the build. Clean up everything to take
advantage of Source().
function is added to the list of files to be built.

Split the syscall table, SPARC specific syscall implementations, and the 32 bit syscall table into it's own file. Corrected problems with the stat structure. These should be tested on 64 bit x86 and SPARC machines.

Initial work to make remote gdb available in SE mode. This is completely untested.

reogranize code to split off FS only misc regs with effect into their own file (reducing the number of if FULL_SYSTEM defines and includes)
Protect other pieces of code so that sparc compiles SE again

src/arch/sparc/SConscript:
Add ua2005.cc back into SConscript
src/arch/sparc/miscregfile.hh:
add functions that deal with priv registers so we don't have to have a bunch of if defs and other ugliness
src/arch/sparc/mmaped_ipr.hh:
wrap handleIpr* with if full_system so it compiles under se
src/arch/sparc/ua2005.cc:
reorganize edit fs only miscreg functions
src/cpu/exetrace.cc:
protect legion code so it doesn't try to compile under se

first cut at a sparc tlb

src/arch/sparc/SConscript:
Add code to serialize/unserialze tlb entries
src/arch/sparc/asi.cc:
src/arch/sparc/asi.hh:
update asi names for how they're listed in the supplement
add asis
add more asi functions
src/arch/sparc/isa_traits.hh:
move the interrupt stuff and some basic address space stuff into isa traits
src/arch/sparc/miscregfile.cc:
src/arch/sparc/miscregfile.hh:
add mmu registers to tlb
get rid of implicit asi stuff... the tlb will handle it
src/arch/sparc/regfile.hh:
make isnt/dataAsid return ints not asis
src/arch/sparc/tlb.cc:
src/arch/sparc/tlb.hh:
first cut at sparc tlb
src/arch/sparc/vtophys.hh:
pagatable nedes to be included here
src/mem/request.hh:
add asi and if the request is a memory mapped register to the requset object
src/sim/host.hh:
fix incorrect definition of LL

Added in alot of missing source files.

removed ua2005.cc since it's been obsorbed into the miscregfile, and added system.cc

Basic work towards supporting ASIs properly

src/arch/sparc/SConscript:
Added a file that implements ASI utility functions. These don't go in utility.hh because they aren't supposed to be part of the generic ISA interface.
src/arch/sparc/asi.hh:
Fixed up some mistranscriptions, and added function prototypes for some ASI utility functions.
src/arch/sparc/asi.cc:
Implementation of some ASI utility functions.

Pushed most of constants.hh back into isa_traits.hh and regfile.hh and created a seperate file for the syscallreturn class.

Updated Authors from bk prs info

commit a couple of minor things that I forgot to last time.

src/SConscript:
src/arch/sparc/SConscript:
commit a couple of things that I forgot to last time.

split off fullsystem and se iprs into two functions to remove lots of #ifs
setup all initialization stuff for UA2005
Setup fullsys build options
Start to make fullsystem compile

src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
RCS to BitKeeper
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Add support for doing virtual to physical translation using the in-memory
page table
src/arch/sparc/vtophys.cc:
a tad bit of error checking
src/arch/sparc/vtophys.hh:
Cleanup of full-system global variables, primarily in simple_cpu.cc,
to allow multi-system simulations. (Multiple systems not yet yested
though.) Also changes to build sim_smt in full-system mode (though
with only SimpleCPU and not the full timing CPU for now).

Still to do: minimize changes in SimpleCPU code between full-system
and application modes... way too many ifdefs there. Much of the
full-system stuff moved into SimpleCPU should be put in a new System
object to allow multiprocessor simulations.

Converted last remaining modules from C to C++ (mostly in /old).
Renamed all .c files to .cc and a few .h files to .hh.

Renamed architecture-specific files in arch/$TARGET from
$TARGET.{cc,hh,def} to machine.{cc,hh,def} to get rid of pointless
intermediate files in object directory. Split exo-specific
definitions out of machine.hh into machine_exo.h.

Specifics:

In machine.def, null resource descriptors must be FUClamd_NA (and not
NA) to pass C++ type checking.

Enhanced error checking/reporting in bas
src/arch/sparc/vtophys.cc:
- Get rid of my String class, the Vector class, the bitvector class, and my
doubly linked list class.
- Convert tokenize, to_number (formerly StringToNumber) and eat_white to
function on stl strings.
- Change most cases of char * and const char * to string, or const string &
- Some formatting and style nits, but not too many.
src/arch/sparc/vtophys.cc:
simplify
src/arch/sparc/vtophys.cc:

Renamed SimpleCPU::(read|write)_(byte|half|word|qword) to just read &
write, overloaded on the type of the 'data' argument. Merged the
full-system and non-full-system implementations of these eight
original functions into two common template functions.

To support this, also renamed (read|write)[1248] on memory_object and
derivatives to just read & write, again overloaded on the type of the
'data' argument. Many of these functions could now be condensed into
a few template functions (though with a level of indirection so that
the interface can remain virtual). I did not do that though.
src/arch/sparc/vtophys.cc:
First pass at compiling with gcc 3.x. Lots of "std::" in header files,
"using namespace std" in source files. (Note policy of not putting "using"
statements in headers or before includes in sources.)

Still not able to compile with gcc 3.2.
Errors:
- Can't create an ifstream from a file descriptor anymore (breaks IniFile).
- "`class MSHR::MSHRegister' is private" errors in mshr.cc and prefetch_cache.cc:
not clear why since it's in the public part of the class declaration.
- cpu.cc:879: can't match a reference and 0 (specifically "no match for `bool ?
SimObjectParam<PipeTrace*>& : int' operator")
- pipetrace.cc: "invalid conversion from `int' to `std::_Ios_Fmtflags'"
Warnings:
- strstream now deprecated... needs some rewriting in sat_counter.hh
and hybrid_pred.hh (need to get all that code out of the headers anyway)
- trace.hh macro problem: cpp now says 'pasting "::" and "Event" does not
give a valid preprocessing token'
- major "implicit typename" issues in base/sized.hh
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Implement some interval statistics for full system mode.
Create a callpal function that is called when a callpal occurs so it's easier
to manipulate the statics.
Rework the vtophys stuff to make it a bit cleaner.
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Get rid of almost all old-style object names. This commit is equivalent
to running the following script on the current head:

#! /bin/sh

find \( -name '*.cc' -o -name '*.hh' \) -exec perl -pi -e '\
s/\bmemory_object\b(?!\.hh)/FunctionalMemory/g;\
s/\bvirtual_memory\b(?!\.hh)/VirtualMemory/g;\
s/\bmain_memory\b(?!\.hh)/MainMemory/g;\
s/\bphysical_memory\b(?!\.hh)/PhysicalMemory/g;\
s/\bspec_memory\b(?!\.hh)/SpeculativeMemory/g;\
s/\bMemObj\b(?!\.hh)/TimingMemObj/g;\
s/\bmemory_translation\b(?!\.hh)/AddressTranslator/g;\
s/\balpha_tlb\b(?!\.hh)/AlphaTlb/g;\
s/\balpha_itb\b(?!\.hh)/AlphaItb/g;\
s/\balpha_dtb\b(?!\.hh)/AlphaDtb/g;\
s/\bmemory_controller\b(?!\.hh)/MemoryController/g;\
s/\bstorebuffer_t\b(?!\.hh)/StoreBuffer/g;\
s/\bstorebuffer_entry_t\b(?!\.hh)/StoreBufferEntry/g;\
s/\bcreate_vector_t\b(?!\.hh)/CreateVector/g;\
s/\bcv_spec_state\b(?!\.hh)/CreateVecSpecState/g;\
s/\bspec_state_list\b(?!\.hh)/SpecStateList/g;\
s/\bdyn_inst_t\b(?!\.hh)/DynInst/g;' {} \;
src/arch/sparc/vtophys.cc:
since cprintf properly deals with 64-bit types, stop using FMT* as much as
possible
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Change byte_t etc. to C99 standard int8_t etc.
Other than old/host.h, all other changes were produced by this script:

#! /bin/sh

find \( -name '*.cc' -o -name '*.hh' -o -name '*.c' -o -name '*.h' -o -name 'machine.def' \) -exec perl -pi -e '\
s/\bbyte_t\b(?!\.hh)/uint8_t/g;\
s/\bsbyte_t\b(?!\.hh)/int8_t/g;\
s/\bhalf_t\b(?!\.hh)/uint16_t/g;\
s/\bshalf_t\b(?!\.hh)/int16_t/g;\
s/\bword_t\b(?!\.hh)/uint32_t/g;\
s/\bsword_t\b(?!\.hh)/int32_t/g;\
s/\bqword_t\b(?!\.hh)/uint64_t/g;\
s/\bsqword_t\b(?!\.hh)/int64_t/g;\
s/\bbool_t\b(?!\.hh)/bool/g;\
s/\bdfloat_t\b(?!\.hh)/double/g;\
s/\bsfloat_t\b(?!\.hh)/float/g;' {} \;
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Add CVS Id tags
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Massive renaming to (almost) eliminate all md_* and MD_* names
in preparation for total exorcism of machine.def.

Most of the changes in this commit were performed with the
following perl script (perl -pi <script> <files>). A small
amount of manual fixup was needed to (mostly getting rid of
the Addr typedefs in the various memory objects now that
the former md_addr_t has that name).

# rename machine-dependent types and constants (will be moving into ISA traits object)
s/md_addr_t/Addr/g;
s/md_intreg_t/IntReg/g;
s/md_gpr_t/IntRegFile/g;
s/md_fpreg_t/FloatReg/g;
s/md_fpr_t/FloatRegFile/g;
s/md_ctrlreg_t/MiscReg/g;
s/md_ctrl_t/MiscRegFile/g;
s/md_ipr_t/InternalProcReg/g;
s/md_anyreg_t/AnyReg/g;
s/md_inst_t/MachInst/g;
s/regs_t/RegFile/g;
# manually fix declaration in old/regs.h and a few forward decls
s/struct RegFile/RegFile/g;
s/MD_NUM_IREGS/NumIntRegs/g;
s/MD_NUM_FREGS/NumFloatRegs/g;
s/MD_NUM_CREGS/NumMiscRegs/g;
s/MD_IPR_NUM/NumInternalProcRegs/g;
s/MD_TOTAL_REGS/TotalNumRegs/g;
s/MD_REG_ZERO/ZeroReg/g;
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
New ISA description system. No more machine.def!
Instructions are now decoded into StaticInst objects, and all static
instruction properties (including execution behavior) are associated
with those objects. Extended documentation in progress.
Currently supports Alpha only; PISA will not compile.
Use END_OF_MACHINE_DOT_DEF tag to extract previous version.
src/arch/sparc/vtophys.cc:
get rid of MD_IPR_foo and call it IPR_foo
add some comments to describe what the various PALtemp registers do
formatting
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
license
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
a little style
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Add attribution to license.
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Make include paths explicit.
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
CopyData and CopyString moved from kernel.cc to vtophys.cc
kernel.cc and kernel.hh moved to kern/tru64
src/arch/sparc/vtophys.hh:
Include isa_traits.hh for Addr
src/arch/sparc/vtophys.cc:
formatting fixes
src/arch/sparc/vtophys.cc:
fix up vtophys to deal with translations if there
is no ptbr, and to deal with PAL addresses
add ptomem which is just a wrapper for dma_addr
src/arch/sparc/vtophys.hh:
add ptomem which is a wrapper for dma_addr with the
same usage as vtomem
src/arch/sparc/vtophys.cc:
Fix to remote debugger while in PAL code
src/arch/sparc/vtophys.cc:
Remote an old hack that is now unnecessary
src/arch/sparc/vtophys.cc:
Removed buggy code that tries to fix PAL addresses (may cause problems
while trying to debug in PAL code, but that should do this fix outside
of vtophys)
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Updated Copyright
src/arch/sparc/vtophys.cc:
added back some code andrew removed and couldn't remember why.
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
rename CopyData to CopyOut and implement CopyIn to copy data
from the simulator into the simulatee
src/arch/sparc/vtophys.cc:
fixed a bad merge from linux<->tru64
src/arch/sparc/vtophys.cc:
Check max address pal can be at so we don't do the wrong conversion
if gdb asks for an unaligned access.
src/arch/sparc/vtophys.cc:
PGOFSET -> ALPHA_PGOFSET to avoid include file problems
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
in the arch/alpha directory we should use arch/alpha, not
targetarch. sort includes while we're here.
src/arch/sparc/vtophys.cc:
use new constants, functions and structs to clean up the
vtophys code.
src/arch/sparc/vtophys.hh:
Clean up a little bit and make the protypes match new changes.
src/arch/sparc/vtophys.cc:
deal with isa addition
src/arch/sparc/vtophys.cc:
shuffle files around for new directory structure
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Remove RCS Id string
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
Update copyright dates and author list
src/arch/sparc/vtophys.cc:
Added a using directive for AlphaISA
src/arch/sparc/vtophys.hh:
Added the AlphaISA namespace specifier where needed
src/arch/sparc/vtophys.hh:
Made Addr a global type
src/arch/sparc/vtophys.cc:
Change access to the IPR to go through the XC.
src/arch/sparc/vtophys.cc:
Avoid directly accessing objects within the XC.
src/arch/sparc/vtophys.cc:
src/arch/sparc/vtophys.hh:
fixed for new memory system
put in namespace AlphaISA
src/arch/alpha/vtophys.cc:
src/arch/alpha/vtophys.hh:
Remove authors from copyright.
src/arch/sparc/vtophys.cc:
bk cp alpha/vtophys.cc sparc/vtophys.cc
src/arch/sparc/vtophys.hh:
bk cp alpha/vtophys.hh sparc/vtophys.hh
src/arch/sparc/SConscript:
remove fullsystem files that don't exist
src/arch/sparc/isa_traits.hh:
split off fullsystem and se iprs into two functions to remove lots of #ifs
src/arch/sparc/regfile.hh:
split off fullsystem and se iprs into two functions to remove lots of #ifs
setup all initialization stuff for UA2005
src/arch/sparc/system.cc:
src/arch/sparc/system.hh:
Add system level tick storage to make stick be syncronized across multiple processors
src/arch/sparc/vtophys.hh:
start to create a vtophys for Sparc
src/base/loader/symtab.hh:
Addr is defined in sim/host.hh

New directory structure:
- simulator source now in 'src' subdirectory
- imported files from 'ext' repository
- support building in arbitrary places, including
outside of the source tree. See comment at top
of SConstruct file for more details.
Regression tests are temporarily disabled; that
syetem needs more extensive revisions.

SConstruct:
Update for new directory structure.
Modify to support build trees that are not subdirectories
of the source tree. See comment at top of file for
more details.
Regression tests are temporarily disabled.
src/arch/SConscript:
src/arch/isa_parser.py:
src/python/SConscript:
Update for new directory structure.