sim: Add a mechanism to exit the simulation loop immediately.

There are some cases, specifically when running systemc, that it's
necessary to exit the simulation loop immediately rather than finishing
running events scheduled for the current Tick. When running under
sc_main, sc_stop and sc_pause return control to sc_main which can
happen immediately. When running without sc_main, control needs to
return to the python config script which needs to happen through a
global exit event.

Since sc_pause and sc_stop are supposed to stop simulation without
necessarily letting all the events at the current time run, we need
a way to schedule an exit event with a very high priority (rather than
a very low priority).

This change adds a new exitSimLoopNow function which does that, and
adds a new constructor to the GlobalSimLoopExitEvent which uses that
priority.

Also, a couple of cruft functions from the sim events are removed.

Change-Id: Icfbec17fb10f98084a75740acd839dbf4096fbb3
Reviewed-on: https://gem5-review.googlesource.com/c/16444
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>

style: [patch 1/22] use /r/3648/ to reorganize includes

sim: Remove autoserialize support for exit events

This changeset removes the support for the autoserialize parameter in
GlobalSimLoopExitEvent (including exitSimLoop()) and
LocalSimLoopExitEvent.

Auto-serialization of the LocalSimLoopExitEvent was never used, so
this is not expected to affect anything. However, it was sometimes
used for GlobalSimLoopExitEvent. Unfortunately, serialization of
global events has never been supported, so checkpoints with such
events will currently cause simulation panics.

The serialize parameter to exitSimLoop() has been left in-place to
maintain API compatibility (removing it would affect m5ops). Instead
of just dropping it, we now print a warning if the parameter is set
and the exit event is scheduled in the future (i.e., not at the
current tick).

sim: Fix broken event unserialization

Events expected to be unserialized using an event-specific
unserializeEvent call. This call was never actually used, which meant
the events relying on it never got unserialized (or scheduled after
unserialization).

Instead of relying on a custom call, we now use the normal
serialization code again. In order to schedule the event correctly,
the parrent object is expected to use the
EventQueue::checkpointReschedule() call. This happens automatically
for events that are serialized using the AutoSerialize mechanism.

sim: Refactor the serialization base class

Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:

* Add a set of APIs to serialize into a subsection of the current
object. Previously, objects that needed this functionality would
use ad-hoc solutions using nameOut() and section name
generation. In the new world, an object that implements the
interface has the methods serializeSection() and
unserializeSection() that serialize into a named /subsection/ of
the current object. Calling serialize() serializes an object into
the current section.

* Move the name() method from Serializable to SimObject as it is no
longer needed for serialization. The fully qualified section name
is generated by the main serialization code on the fly as objects
serialize sub-objects.

* Add a scoped ScopedCheckpointSection helper class. Some objects
need to serialize data structures, that are not deriving from
Serializable, into subsections. Previously, this was done using
nameOut() and manual section name generation. To simplify this,
this changeset introduces a ScopedCheckpointSection() helper
class. When this class is instantiated, it adds a new /subsection/
and subsequent serialization calls during the lifetime of this
helper class happen inside this section (or a subsection in case
of nested sections).

* The serialize() call is now const which prevents accidental state
manipulation during serialization. Objects that rely on modifying
state can use the serializeOld() call instead. The default
implementation simply calls serialize(). Note: The old-style calls
need to be explicitly called using the
serializeOld()/serializeSectionOld() style APIs. These are used by
default when serializing SimObjects.

* Both the input and output checkpoints now use their own named
types. This hides underlying checkpoint implementation from
objects that need checkpointing and makes it easier to change the
underlying checkpoint storage code.

sim: simulate with multiple threads and event queues
This patch adds support for simulating with multiple threads, each of
which operates on an event queue. Each sim object specifies which eventq
is would like to be on. A custom barrier implementation is being added
using which eventqs synchronize.

The patch was tested in two different configurations:
1. ruby_network_test.py: in this simulation L1 cache controllers receive
requests from the cpu. The requests are replied to immediately without
any communication taking place with any other level.
2. twosys-tsunami-simple-atomic: this configuration simulates a client-server
system which are connected by an ethernet link.

We still lack the ability to communicate using message buffers or ports. But
other things like simulation start and end, synchronizing after every quantum
are working.

Committed by: Nilay Vaish

sim: added option to serialize SimLoopExitEvent

SimLoopExitEvents weren't serialized by default. Some benchmarks
utilize a delayed m5 exit pseudo op call to terminate the simulation
and this event was lost when resuming from a checkpoint generated
after the pseudo op call. This patch adds the capability to serialize
the SimLoopExitEvents and enable serialization for m5_exit and m5_fail
pseudo ops by default. Does not affect other generic
SimLoopExitEvents.

sim: have a curTick per eventq
This patch adds a _curTick variable to an eventq. This variable is updated
whenever an event is serviced in function serviceOne(), or all events upto
a particular time are processed in function serviceEvents(). This change
helps when there are eventqs that do not make use of curTick for scheduling
events.

sim: Move the draining interface into a separate base class

This patch moves the draining interface from SimObject to a separate
class that can be used by any object needing draining. However,
objects not visible to the Python code (i.e., objects not deriving
from SimObject) still depend on their parents informing them when to
drain. This patch also gets rid of the CountedDrainEvent (which isn't
really an event) and replaces it with a DrainManager.

sim: Fix as issue where exit events on instr queues are used after freed.

event: minor cleanup
Initialize flags via the Event constructor instead of calling
setFlags() in the body of the derived class's constructor. I
forget exactly why, but this made life easier when implementing
multi-queue support.

Also rename Event::getFlags() to isFlagSet() to better match
common usage, and get rid of some unused Event methods.

Replace curTick global variable with accessor functions.
This step makes it easy to replace the accessor functions
(which still access a global variable) with ones that access
per-thread curTick values.

sim: clean up CountedDrainEvent slightly.
There's no reason for it to derive from SimLoopExitEvent.
This whole drain thing needs to be redone eventually,
but this is a stopgap to make later changes to
SimLoopExitEvent feasible.

sim: delete unused CheckSwapEvent code.
There's no way to even create one of these anymore.

pseudoinst: get rid of mainEventQueue references.
Avoid direct references to mainEventQueue in pseudo-insts
by indirecting through associated CPU object.
Made exitSimLoop() more flexible to enable some of these.

sim: fold StartupCallback into SimObject
There used to be a reason to have StartupCallback
be a separate object, but not any more. Now
it's just confusing.

eventq: convert all usage of events to use the new API.
For now, there is still a single global event queue, but this is
necessary for making the steps towards a parallelized m5.

eventq: Clean up the Event class so that it uses fewer bytes. This
will hopefullly allow it to fit in a cache line.

Make the Event::description() a const function

Major changes to how SimObjects are created and initialized. Almost all
creation and initialization now happens in python. Parameter objects
are generated and initialized by python. The .ini file is now solely for
debugging purposes and is not used in construction of the objects in any
way.

Move all of the parameters of the Root SimObject so they are
directly configured by python. Move stuff from root.(cc|hh) to
core.(cc|hh) since it really belogs there now.
In the process, simplify how ticks are used in the python code.

there are two main thrusts of this changeset.

1) return the periodicity of checkpoints back into the code (i.e. make m5 checkpoint n m meaningful again).
2) to do this, i had to much around with being able to repeatedly schedule and SimLoopExitEvent, which led to changes in how exit simloop events are handled to make this easier.

src/arch/alpha/isa/decoder.isa:
src/mem/cache/cache_impl.hh:
modify arg. order for new calling convention of exitSimLoop.
src/cpu/base.cc:
src/sim/main.cc:
src/sim/pseudo_inst.cc:
src/sim/root.cc:
now, instead of creating a new SimLoopExitEvent, call a wrapper schedExitSimLoop which handles all the default args.
src/sim/sim_events.cc:
src/sim/sim_events.hh:
src/sim/sim_exit.hh:
add the periodicity of checkpointing back into the code.

to facilitate this, there are now two wrappers (instead of just overloading exitSimLoop). exitSimLoop is only for exiting NOW (i.e. at curTick), while schedExitSimLoop schedules and exit event for the future.

Rename quiesce to drain to avoid confusion with the pseudo instruction.

src/cpu/simple/timing.cc:
src/cpu/simple/timing.hh:
src/python/m5/__init__.py:
src/python/m5/config.py:
src/sim/main.cc:
src/sim/sim_events.cc:
src/sim/sim_events.hh:
src/sim/sim_object.cc:
src/sim/sim_object.hh:
Rename quiesce to drain.

Add in support for quiescing the system, taking checkpoints, restoring from checkpoints, changing memory modes, and switching CPUs.

Key new functions that can be called on the m5 object at the python interpreter:
doQuiesce(root) - A helper function that quiesces the object passed in and all of its children.
resume(root) - Another helper function that tells the object and all of its children that the quiesce is over.
checkpoint(root) - Takes a checkpoint of the system. Checkpoint directory must be set before hand.
setCheckpointDir(name) - Sets the checkpoint directory.
restoreCheckpoint(root) - Restores the values from the checkpoint located in the checkpoint directory.
changeToAtomic(system) - Changes the system and all of its children to atomic memory mode.
changeToTiming(system) - Changes the system and all of its children to timing memory mode.
switchCpus(list) - Takes in a list of tuples, where each tuple is a pair of (old CPU, new CPU). Quiesces the old CPUs, and then switches over to the new CPUs.

src/SConscript:
Remove serializer, replaced by python code.
src/python/m5/__init__.py:
Updates to support quiescing, checkpointing, changing memory modes, and switching CPUs.
src/python/m5/config.py:
Several functions defined on the SimObject for quiescing, changing timing modes, and switching CPUs
src/sim/main.cc:
Add some extra functions that are exported to python through SWIG.
src/sim/serialize.cc:
Change serialization around a bit. Now it is controlled through Python, so there's no need for SerializeEvents or SerializeParams.

Also add in a new unserializeAll() function that loads a checkpoint and handles unserializing all objects.
src/sim/serialize.hh:
Add unserializeAll function and a setCheckpointName function.
src/sim/sim_events.cc:
Add process() function for CountedQuiesceEvent, which calls exitSimLoop() once its counter reaches 0.
src/sim/sim_events.hh:
Add in a CountedQuiesceEvent, which is used when the system is preparing to quiesce. Any objects that can't be quiesced immediately are given a pointer to a CountedQuiesceEvent. The event has its counter set via Python, and as objects finish quiescing they call process() on the event. Eventually the event causes the simulation to stop once all objects have quiesced.
src/sim/sim_object.cc:
Add a few functions for quiescing, checkpointing, and changing memory modes.
src/sim/sim_object.hh:
Add a state variable to all SimObjects that tracks both the timing mode of the object and the quiesce state of the object. Currently this isn't serialized, and I'm not sure it needs to be so long as the timing mode starts up the same after a checkpoint.

Move main control from C++ into Python.
User script now invokes initialization and
simulation loop after building configuration.
These functions are exported from C++ to Python
using SWIG.

SConstruct:
Set up SWIG builder & scanner.
Set up symlinking of source files into build directory
(by not disabling the default behavior).
configs/test/test.py:
Rewrite to use new script-driven interface.
Include a sample option.
src/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
Add SWIG-generated main_wrap.cc to source list.
src/arch/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
src/arch/alpha/ev5.cc:
src/arch/alpha/isa/decoder.isa:
src/cpu/o3/alpha_cpu_impl.hh:
src/cpu/trace/opt_cpu.cc:
src/cpu/trace/trace_cpu.cc:
src/sim/pseudo_inst.cc:
src/sim/root.cc:
src/sim/serialize.cc:
src/sim/syscall_emul.cc:
SimExit() is now exitSimLoop().
src/cpu/base.cc:
SimExitEvent is now SimLoopExitEvent
src/python/SConscript:
Add SWIG build command for main.i.
Use python/m5 in build dir as source for zip archive...
easy now with file duplication enabled.
src/python/m5/__init__.py:
- Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
- Get rid of argument parsing code; just provide default
option descriptors for user script to call optparse with.
- Don't clutter m5 namespace by sucking in all of m5.config
and m5.objects.
- Move instantiate() function here from config.py.
src/python/m5/config.py:
- Move instantiate() function to __init__.py.
- Param.Foo deferred type lookups must use m5.objects
namespace now (not m5).
src/python/m5/objects/AlphaConsole.py:
src/python/m5/objects/AlphaFullCPU.py:
src/python/m5/objects/AlphaTLB.py:
src/python/m5/objects/BadDevice.py:
src/python/m5/objects/BaseCPU.py:
src/python/m5/objects/BaseCache.py:
src/python/m5/objects/Bridge.py:
src/python/m5/objects/Bus.py:
src/python/m5/objects/CoherenceProtocol.py:
src/python/m5/objects/Device.py:
src/python/m5/objects/DiskImage.py:
src/python/m5/objects/Ethernet.py:
src/python/m5/objects/Ide.py:
src/python/m5/objects/IntrControl.py:
src/python/m5/objects/MemObject.py:
src/python/m5/objects/MemTest.py:
src/python/m5/objects/Pci.py:
src/python/m5/objects/PhysicalMemory.py:
src/python/m5/objects/Platform.py:
src/python/m5/objects/Process.py:
src/python/m5/objects/Repl.py:
src/python/m5/objects/Root.py:
src/python/m5/objects/SimConsole.py:
src/python/m5/objects/SimpleDisk.py:
src/python/m5/objects/System.py:
src/python/m5/objects/Tsunami.py:
src/python/m5/objects/Uart.py:
Fix up imports (m5 namespace no longer includes m5.config).
src/sim/eventq.cc:
src/sim/eventq.hh:
Support for Python-called simulate() function:
- Use IsExitEvent flag to signal events that want
to exit the simulation loop gracefully (instead of
calling exit() to terminate the process).
- Modify interface to hand exit event object back to
caller so it can be inspected for cause.
src/sim/host.hh:
Add MaxTick constant.
src/sim/main.cc:
Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
Use PYTHONPATH environment var to set module path
(instead of clunky code injection method).
Move main control from here into Python:
- Separate initialization code and simulation loop
into separate functions callable from Python.
- Make Python interpreter invocation more pure (more
like directly invoking interpreter).
Add -i and -p flags (only options on binary itself;
other options processed by Python).
Import readline package when using interactive mode.
src/sim/sim_events.cc:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
src/sim/sim_events.hh:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
Get rid of a few unused constructors.
src/sim/sim_exit.hh:
SimExit() is now exitSimLoop().
Get rid of unused functions.
Add comments.

Updated Authors from bk prs info

New directory structure:
- simulator source now in 'src' subdirectory
- imported files from 'ext' repository
- support building in arbitrary places, including
outside of the source tree. See comment at top
of SConstruct file for more details.
Regression tests are temporarily disabled; that
syetem needs more extensive revisions.

SConstruct:
Update for new directory structure.
Modify to support build trees that are not subdirectories
of the source tree. See comment at top of file for
more details.
Regression tests are temporarily disabled.
src/arch/SConscript:
src/arch/isa_parser.py:
src/python/SConscript:
Update for new directory structure.