systemc: Add m5.systemc and m5.tlm python modules.

These will be how systemc and tlm APIs which are not attached to
SimObjects will be exposed. This avoids having to artificially attach
them to wrapping SimObjects for instance, which is a bit awkward
and non-obvious.

The python code which attaches the systemc and tlm modules to the
m5 modules lives in src/python/m5/__init__.py, but the modules
themselves live in src/systemc/python to keep all the systemc code
grouped together. It might be a little confusing to have a small part
of the glue that adds those modules in a separate place (__init__.py),
but that is, as far as I can tell, unavoidable, and it's better in my
opinion to keep the systemc code grouped together than to put it
alongside the other python code and __init__.py.

Change-Id: Iecb218daec5e15772152b5ad22b51f43b86c3d4b
Reviewed-on: https://gem5-review.googlesource.com/c/16563
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Gabe Black <gabeblack@google.com>

python: Enforce absolute imports for Python 3 compatibility

Change-Id: Ia88d7fd472f7aed9b97df81468211384981bf6c6
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/15983
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>

python: Remove SWIG

Remove SWIG-specific Python code.

Change-Id: If1d1b253d84021c9a8f9a64027ea7a94f2336dff
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2922
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>

python: Move native wrappers to the _m5 namespace

Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.

Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

swig: Fix issue with circular import in 2.0.9/2.0.10

This patch fixes an issue which prevented gem5 from running when built
using swig 2.0.9 and 2.0.10. The generated event.py tried to import
m5.internal which in turn relied on importing event. This patch seems
to fix the problem, and so far has not caused any other issues.

sim: Move the draining interface into a separate base class

This patch moves the draining interface from SimObject to a separate
class that can be used by any object needing draining. However,
objects not visible to the Python code (i.e., objects not deriving
from SimObject) still depend on their parents informing them when to
drain. This patch also gets rid of the CountedDrainEvent (which isn't
really an event) and replaces it with a DrainManager.

python: different import for dealing with demandimport

python: cleanup python code so stuff doesn't automatically happen at startup
this allows things to be overridden at startup (e.g. for tests)

python: figure out if the m5.internal package exists even with demandimport

python: Move more code into m5.util allow SCons to use that code.
Get rid of misc.py and just stick misc things in __init__.py
Move utility functions out of SCons files and into m5.util
Move utility type stuff from m5/__init__.py to m5/util/__init__.py
Remove buildEnv from m5 and allow access only from m5.defines
Rename AddToPath to addToPath while we're moving it to m5.util
Rename read_command to readCommand while we're moving it
Rename compare_versions to compareVersions while we're moving it.

python: Make it possible to import the parts of m5 that are pure python

Errors: Print a URL with a hash of the format string to find more information about an error.

python: add fatal() function to the m5 package and use it

python: Try to isolate the stuff that's in the m5.internal package a bit more.

python: Rework how things are imported

libm5: Create a libm5 static library for embedding m5.

This should allow m5 to be more easily embedded into other simulators.
The m5 binary adds a simple main function which then calls into the m5
libarary to start the simulation. In order to make this work
correctly, it was necessary embed python code directly into the
library instead of the zipfile hack. This is because you can't just
append the zipfile to the end of a library the way you can a binary.
As a result, Python files that are part of the m5 simulator are now
compile, marshalled, compressed, and then inserted into the library's
data section with a certain symbol name. Additionally, a new Importer
was needed to allow python to get at the embedded python code.

Small additional changes include:
- Get rid of the PYTHONHOME stuff since I don't think anyone ever used
it, and it just confuses things. Easy enough to add back if I'm wrong.
- Create a few new functions that are key to initializing and running
the simulator: initSignals, initM5Python, m5Main.

The original code for creating libm5 was inspired by a patch Michael
Adler, though the code here was done by me.

python: provide access to stats

python: fix m5.build_env variable.
As it is now, some objects will get the incorrect value depending where they
were defined.

Major changes to how SimObjects are created and initialized. Almost all
creation and initialization now happens in python. Parameter objects
are generated and initialized by python. The .ini file is now solely for
debugging purposes and is not used in construction of the objects in any
way.

Add a function to get a SimObject's memory mode and rework
the set memory mode code to only go through the change if
it is necessary

Cleanup

Move all of the parameters of the Root SimObject so they are
directly configured by python. Move stuff from root.(cc|hh) to
core.(cc|hh) since it really belogs there now.
In the process, simplify how ticks are used in the python code.

Python atexit handlers are called in reverse order.
Fix things so the stats dump happens last.

Factor code out of main.cc and main.i into a bunch of files
so things are organized in a more sensible manner. Take apart
finalInit and expose the individual functions which are now
called from python. Make checkpointing a bit easier to use.

Create a module called internal where swigged stuff goes.
Rename cc_main to internal.main

simplify maxtick parsing in both the python and the c++.

configs/common/Simulation.py:
simplify maxtick code a little bit - instead of checking for -1, just set it at MaxTick.
src/python/m5/__init__.py:
make a new m5 param called MaxTick.
src/sim/host.hh:
fix the M5 def. of MaxTick
src/sim/main.cc:
Simplify the MaxTick/num_cycles parsing within main.cc

Fix error message.

get rid of the "resume" step at the end of changeToTiming/Atomic because this will cause an assertion when you do the CPU switch. instead, push the responsibility of the resume upwards towards the user - documented in se.py and fs.py so it should be ok.

System not global object, need to preface it with objects.

Enable proxies (Self/Parent) for specifying ports.
Significant revamp of Port code.
Some cleanup of SimObject code too, particularly to
make the SimObject and MetaSimObject implementations of
__setattr__ more consistent.
Unproxy code split out of print_ini().

src/python/m5/multidict.py:
Make get() return None by default, to match semantics
of built-in dictionary objects.

More Python hacking to deal with config.py split
and resulting recursive import trickiness.

Split config.py into multiple files.
Some tweaking to deal with mutually recursive imports.

Clean up some more config stuff.

configs/common/FSConfig.py:
Clean up some code to make functions look less like classes. Also put makeList function (formerly listWrapper) into m5 itself.
configs/test/fs.py:
Update for changed code.
src/python/m5/__init__.py:
Put makeList into m5.

Migrate most of main() and and all option parsing to python

configs/test/fs.py:
configs/test/test.py:
update for the new way that m5 deals with options
src/python/SConscript:
Compile AUTHORS, LICENSE, README, and RELEASE_NOTES into the
python stuff.
src/python/m5/__init__.py:
redo the way options work.
Move them all to main.py
src/sim/main.cc:
Migrate more functionality for main() into python.
Namely option parsing
src/python/m5/attrdict.py:
A dictionary object that overrides attribute access to
do item access.
src/python/m5/main.py:
The new location for M5's option parsing, and the main()
routine to set up the simulation.

Take the name of the checkpoint directory in when calling checkpoint() or restoreCheckpoint().

src/sim/main.cc:
src/sim/serialize.cc:
src/sim/serialize.hh:
Take in the directory name when checkpointing.

Be sure to call resume after restoring from a checkpoint.

Fixes for draining.

src/cpu/simple/timing.cc:
Update for changed return values.
src/python/m5/__init__.py:
Loop in order to make sure all objects are really drained. Objects may become undrained as other objects become drained (e.g. a bus-bridge has a packet, while a bus is empty, and the first drain() will cause the bus-bridge to give the packet to the bus).

The only case we know every object is actually drained is if they all return immediately that they are drained.

Rename quiesce to drain to avoid confusion with the pseudo instruction.

src/cpu/simple/timing.cc:
src/cpu/simple/timing.hh:
src/python/m5/__init__.py:
src/python/m5/config.py:
src/sim/main.cc:
src/sim/sim_events.cc:
src/sim/sim_events.hh:
src/sim/sim_object.cc:
src/sim/sim_object.hh:
Rename quiesce to drain.

Add in support for quiescing the system, taking checkpoints, restoring from checkpoints, changing memory modes, and switching CPUs.

Key new functions that can be called on the m5 object at the python interpreter:
doQuiesce(root) - A helper function that quiesces the object passed in and all of its children.
resume(root) - Another helper function that tells the object and all of its children that the quiesce is over.
checkpoint(root) - Takes a checkpoint of the system. Checkpoint directory must be set before hand.
setCheckpointDir(name) - Sets the checkpoint directory.
restoreCheckpoint(root) - Restores the values from the checkpoint located in the checkpoint directory.
changeToAtomic(system) - Changes the system and all of its children to atomic memory mode.
changeToTiming(system) - Changes the system and all of its children to timing memory mode.
switchCpus(list) - Takes in a list of tuples, where each tuple is a pair of (old CPU, new CPU). Quiesces the old CPUs, and then switches over to the new CPUs.

src/SConscript:
Remove serializer, replaced by python code.
src/python/m5/__init__.py:
Updates to support quiescing, checkpointing, changing memory modes, and switching CPUs.
src/python/m5/config.py:
Several functions defined on the SimObject for quiescing, changing timing modes, and switching CPUs
src/sim/main.cc:
Add some extra functions that are exported to python through SWIG.
src/sim/serialize.cc:
Change serialization around a bit. Now it is controlled through Python, so there's no need for SerializeEvents or SerializeParams.

Also add in a new unserializeAll() function that loads a checkpoint and handles unserializing all objects.
src/sim/serialize.hh:
Add unserializeAll function and a setCheckpointName function.
src/sim/sim_events.cc:
Add process() function for CountedQuiesceEvent, which calls exitSimLoop() once its counter reaches 0.
src/sim/sim_events.hh:
Add in a CountedQuiesceEvent, which is used when the system is preparing to quiesce. Any objects that can't be quiesced immediately are given a pointer to a CountedQuiesceEvent. The event has its counter set via Python, and as objects finish quiescing they call process() on the event. Eventually the event causes the simulation to stop once all objects have quiesced.
src/sim/sim_object.cc:
Add a few functions for quiescing, checkpointing, and changing memory modes.
src/sim/sim_object.hh:
Add a state variable to all SimObjects that tracks both the timing mode of the object and the quiesce state of the object. Currently this isn't serialized, and I'm not sure it needs to be so long as the timing mode starts up the same after a checkpoint.

Merge ktlim@zizzer:/bk/newmem
into zamp.eecs.umich.edu:/z/ktlim2/clean/newmem-merge

Change options back to just being flags instead of taking in a True/False value.

src/python/m5/__init__.py:
Change up options. Now setting the flag enables/disables, each of which is the opposite of the default values found in the Python class.

Rename SWIG "main" module to "cc_main" so it's
clear from the Python side that this is the
interface to C++.

src/SConscript:
main_wrap.cc -> cc_main_wrap.cc
src/python/SConscript:
src/python/m5/__init__.py:
src/sim/main.cc:
s/main/cc_main/
src/python/m5/config.py:
s/main/cc_main/
Also directly import cc_main so we don't need
to put the "m5." in front all the time.

Add --outdir option. Didn't call it "-d" since
that's already being used for "detailed cpu".
Needed to add extra function for user script
to pass parsed options back to m5 module.

configs/test/fs.py:
configs/test/test.py:
Call setStandardOptions().
src/python/m5/__init__.py:
Add --outdir option.
Add setStandardOptions() so user script can
pass parsed options back to m5 module.
src/sim/main.cc:
Add SWIG-wrappable function to set output dir.

Add in some of the commonly used Trace/ExeTrace/Debug options.

src/python/m5/__init__.py:
Add in some of the commonly used Trace/ExeTrace/Debug options. Not terribly clean but it works.

Get Port stuff working with full-system scripts.
Key was adding support for cloning port references (trickier than it sounds).
Got rid of class/instance thing and go back to instance cloning...
still don't allow changing SimObject parameters/children after a
class (instance) has been subclassed or instantiated (or cloned), which
should avoid bizarre unintended behavior.

configs/test/fs.py:
Add ".port" to busses to get a port reference.
Get rid of commented-out code.
src/python/m5/__init__.py:
resolveSimObject should call getCCObject() instead of createCCObject()
to avoid cycles in recursively creating objects.
src/python/m5/config.py:
Get rid of class/instance thing and go back to instance cloning.
Deep copy has to happen only on instance cloning then (and not on subclassing).
Add getCCObject() method to force creation of C++ SimObject without
recursively creating its children.
Add support for cloning port references (trickier than it sounds).
Also clean up some very obsolete comments.
src/python/m5/objects/Bridge.py:
src/python/m5/objects/Device.py:
Add ports.

Merge zizzer:/bk/newmem
into vm1.(none):/home/stever/bk/newmem

Move SimObject creation and Port connection loops
into Python.
Add Port and VectorPort objects and support for
specifying port connections via assignment.
The whole C++ ConfigNode hierarchy is gone now, as are
C++ Connector objects.

configs/test/fs.py:
configs/test/test.py:
Rewrite for new port connector syntax.
src/SConscript:
Remove unneeded files:
- mem/connector.*
- sim/config*
src/dev/io_device.hh:
src/mem/bridge.cc:
src/mem/bridge.hh:
src/mem/bus.cc:
src/mem/bus.hh:
src/mem/mem_object.hh:
src/mem/physical.cc:
src/mem/physical.hh:
Allow getPort() to take an optional index to
support vector ports (eventually).
src/python/m5/__init__.py:
Move SimObject construction and port connection
operations into Python (with C++ calls).
src/python/m5/config.py:
Move SimObject construction and port connection
operations into Python (with C++ calls).
Add support for declaring and connecting MemObject
ports in Python.
src/python/m5/objects/Bus.py:
src/python/m5/objects/PhysicalMemory.py:
Add port declaration.
src/sim/builder.cc:
src/sim/builder.hh:
src/sim/serialize.cc:
src/sim/serialize.hh:
ConfigNodes are gone; builder just gets the
name of a .ini file section now.
src/sim/main.cc:
Move SimObject construction and port connection
operations into Python (with C++ calls).
Split remaining initialization operations into two parts,
loadIniFile() and finalInit().
src/sim/param.cc:
src/sim/param.hh:
SimObject resolution done globally in Python now
(not via ConfigNode hierarchy).
src/sim/sim_object.cc:
Remove unneeded #include.

Add in a few global options. Feel free to rename them, they're just the first thing that came to mind.

src/python/m5/__init__.py:
Add in a few global options.

Merge vm1.(none):/home/stever/bk/newmem
into vm1.(none):/home/stever/bk/newmem-py

src/python/m5/__init__.py:
src/sim/syscall_emul.cc:
Hand merge.

Same fix for -P option as Steve's previous fix.

Fix --var=val arg handling.

Move main control from C++ into Python.
User script now invokes initialization and
simulation loop after building configuration.
These functions are exported from C++ to Python
using SWIG.

SConstruct:
Set up SWIG builder & scanner.
Set up symlinking of source files into build directory
(by not disabling the default behavior).
configs/test/test.py:
Rewrite to use new script-driven interface.
Include a sample option.
src/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
Add SWIG-generated main_wrap.cc to source list.
src/arch/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
src/arch/alpha/ev5.cc:
src/arch/alpha/isa/decoder.isa:
src/cpu/o3/alpha_cpu_impl.hh:
src/cpu/trace/opt_cpu.cc:
src/cpu/trace/trace_cpu.cc:
src/sim/pseudo_inst.cc:
src/sim/root.cc:
src/sim/serialize.cc:
src/sim/syscall_emul.cc:
SimExit() is now exitSimLoop().
src/cpu/base.cc:
SimExitEvent is now SimLoopExitEvent
src/python/SConscript:
Add SWIG build command for main.i.
Use python/m5 in build dir as source for zip archive...
easy now with file duplication enabled.
src/python/m5/__init__.py:
- Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
- Get rid of argument parsing code; just provide default
option descriptors for user script to call optparse with.
- Don't clutter m5 namespace by sucking in all of m5.config
and m5.objects.
- Move instantiate() function here from config.py.
src/python/m5/config.py:
- Move instantiate() function to __init__.py.
- Param.Foo deferred type lookups must use m5.objects
namespace now (not m5).
src/python/m5/objects/AlphaConsole.py:
src/python/m5/objects/AlphaFullCPU.py:
src/python/m5/objects/AlphaTLB.py:
src/python/m5/objects/BadDevice.py:
src/python/m5/objects/BaseCPU.py:
src/python/m5/objects/BaseCache.py:
src/python/m5/objects/Bridge.py:
src/python/m5/objects/Bus.py:
src/python/m5/objects/CoherenceProtocol.py:
src/python/m5/objects/Device.py:
src/python/m5/objects/DiskImage.py:
src/python/m5/objects/Ethernet.py:
src/python/m5/objects/Ide.py:
src/python/m5/objects/IntrControl.py:
src/python/m5/objects/MemObject.py:
src/python/m5/objects/MemTest.py:
src/python/m5/objects/Pci.py:
src/python/m5/objects/PhysicalMemory.py:
src/python/m5/objects/Platform.py:
src/python/m5/objects/Process.py:
src/python/m5/objects/Repl.py:
src/python/m5/objects/Root.py:
src/python/m5/objects/SimConsole.py:
src/python/m5/objects/SimpleDisk.py:
src/python/m5/objects/System.py:
src/python/m5/objects/Tsunami.py:
src/python/m5/objects/Uart.py:
Fix up imports (m5 namespace no longer includes m5.config).
src/sim/eventq.cc:
src/sim/eventq.hh:
Support for Python-called simulate() function:
- Use IsExitEvent flag to signal events that want
to exit the simulation loop gracefully (instead of
calling exit() to terminate the process).
- Modify interface to hand exit event object back to
caller so it can be inspected for cause.
src/sim/host.hh:
Add MaxTick constant.
src/sim/main.cc:
Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
Use PYTHONPATH environment var to set module path
(instead of clunky code injection method).
Move main control from here into Python:
- Separate initialization code and simulation loop
into separate functions callable from Python.
- Make Python interpreter invocation more pure (more
like directly invoking interpreter).
Add -i and -p flags (only options on binary itself;
other options processed by Python).
Import readline package when using interactive mode.
src/sim/sim_events.cc:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
src/sim/sim_events.hh:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
Get rid of a few unused constructors.
src/sim/sim_exit.hh:
SimExit() is now exitSimLoop().
Get rid of unused functions.
Add comments.

Updated Authors from bk prs info

Link in Python interpreter.
Use embedded zip archive to carry Python code instead
of homegrown embedded string/file mechanism.
Do argument parsing in Python instead of C++.

SConstruct:
Add Python interpreter include path & library.
Define two new simple builders which copy &
concatenate files, respectively, for use by
the Python embedded zipfile code.
src/SConscript:
Encapsulate environment creation in a function.
Add code to append Python zip archive to final executable.
Eliminate references to obsolete files.
src/python/SConscript:
Rewrite to generate embedded zip archive of Python code
(replacing old "embedded string" mechanism).
src/python/m5/__init__.py:
Move main arg-parsing loop here (out of C++ main()).
src/python/m5/config.py:
Minor fix (version incompatibility?).
src/sim/main.cc:
Invoke embedded Python interpreter to parse args
and generate config.ini, replacing C++ arg parsing code.

New directory structure:
- simulator source now in 'src' subdirectory
- imported files from 'ext' repository
- support building in arbitrary places, including
outside of the source tree. See comment at top
of SConstruct file for more details.
Regression tests are temporarily disabled; that
syetem needs more extensive revisions.

SConstruct:
Update for new directory structure.
Modify to support build trees that are not subdirectories
of the source tree. See comment at top of file for
more details.
Regression tests are temporarily disabled.
src/arch/SConscript:
src/arch/isa_parser.py:
src/python/SConscript:
Update for new directory structure.