tests: Update test scripts to work with Python 3

Change-Id: I71b1e595765fed9e9f234c9722c33ac5348d4f11
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/15999
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>

tests: Don't override tick rate in Ruby tests

Most Ruby tests assume that the highest frequency in the system under
test is 1GHz and limits the global tick rate to this frequency. This
assumption is broken since the default Ruby configuration scripts
clock the CPU at 2Ghz, which results in warnings and sometimes
incorrect behaviour.

Change-Id: I4b204660862ce3b0ea4a13df42caacd4398fef8c
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/15975
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

config: Make configs/common a Python package

Continue along the same line as the recent patch that made the
Ruby-related config scripts Python packages and make also the
configs/common directory a package.

All affected config scripts are updated (hopefully).

Note that this change makes it apparent that the current organisation
and naming of the config directory and its subdirectories is rather
chaotic. We mix scripts that are directly invoked with scripts that
merely contain convenience functions. While it is not addressed in
this patch we should follow up with a re-organisation of the
config structure, and renaming of some of the packages.

ruby: Fix regressions and make Ruby configs Python packages

This patch moves the addition of network options into the Ruby module
to avoid the regressions all having to add it explicitly. Doing this
exposes an issue in our current config system though, namely the fact
that addtoPath is relative to the Python script being executed. Since
both example and regression scripts use the Ruby module we would end
up with two different (relative) paths being added. Instead we take a
first step at turning the config modules into Python packages, simply
by adding a __init__.py in the configs/ruby, configs/topologies and
configs/network subdirectories.

As a result, we can now add the top-level configs directory to the
Python search path, and then use the package names in the various
modules. The example scripts are also updated, and the messy
path-deducing variations in the scripts are unified.

style: remove trailing whitespace

Result of running 'hg m5style --skip-all --fix-white -a'.

cpu: Tidy up the MemTest and make false sharing more obvious

The MemTest class really only tests false sharing, and as such there
was a lot of old cruft that could be removed. This patch cleans up the
tester, and also makes it more clear what the assumptions are. As part
of this simplification the reference functional memory is also
removed.

The regression configs using MemTest are updated to reflect the
changes, and the stats will be bumped in a separate patch. The example
config will be updated in a separate patch due to more extensive
re-work.

In a follow-on patch a new tester will be introduced that uses the
MemChecker to implement true sharing.

ruby: interface with classic memory controller
This patch is the final in the series. The whole series and this patch in
particular were written with the aim of interfacing ruby's directory controller
with the memory controller in the classic memory system. This is being done
since ruby's memory controller has not being kept up to date with the changes
going on in DRAMs. Classic's memory controller is more up to date and
supports multiple different types of DRAM. This also brings classic and
ruby ever more close. The patch also changes ruby's memory controller to
expose the same interface.

ruby: single physical memory in fs mode
Both ruby and the system used to maintain memory copies. With the changes
carried for programmed io accesses, only one single memory is required for
fs simulations. This patch sets the copy of memory that used to reside
with the system to null, so that no space is allocated, but address checks
can still be carried out. All the memory accesses now source and sink values
to the memory maintained by ruby.

mem: Rename Bus to XBar to better reflect its behaviour

This patch changes the name of the Bus classes to XBar to better
reflect the actual timing behaviour. The actual instances in the
config scripts are not renamed, and remain as e.g. iobus or membus.

As part of this renaming, the code has also been clean up slightly,
making use of range-based for loops and tidying up some comments. The
only changes outside the bus/crossbar code is due to the delay
variables in the packet.

config: ruby: rename _cpu_ruby_ports to _cpu_ports

ruby: add option for number of transitions per cycle
The number of transitions per cycle that a controller can carry out is
a proxy for the number of ports that a controller has. This value is
currently 32 which is way too high. The patch introduces an option
for the number of ports and uses this option in the protocol files
to set the number of transitions. The default value is being set to
4. None of the se regressions change. Ruby stats for the fs regression
change and are being updated.

power: Add voltage domains to the clock domains

This patch adds the notion of voltage domains, and groups clock
domains that operate under the same voltage (i.e. power supply) into
domains. Each clock domain is required to be associated with a voltage
domain, and the latter requires the voltage to be explicitly set.

A voltage domain is an independently controllable voltage supply being
provided to section of the design. Thus, if you wish to perform
dynamic voltage scaling on a CPU, its clock domain should be
associated with a separate voltage domain.

The current implementation of the voltage domain does not take into
consideration cases where there are derived voltage domains running at
ratio of native voltage domains, as with the case where there can be
on-chip buck/boost (charge pumps) voltage regulation logic.

The regression and configuration scripts are updated with a generic
voltage domain for the system, and one for the CPUs.

config: Move the memory instantiation outside FSConfig

This patch moves the instantiation of the memory controller outside
FSConfig and instead relies on the mem_ranges to pass the information
to the caller (e.g. fs.py or one of the regression scripts). The main
motivation for this change is to expose the structural composition of
the memory system and allow more tuning and configuration without
adding a large number of options to the makeSystem functions.

The patch updates the relevant example scripts to maintain the current
functionality. As the order that ports are connected to the memory bus
changes (in certain regresisons), some bus stats are shuffled
around. For example, what used to be layer 0 is now layer 1.

Going forward, options will be added to support the addition of
multi-channel memory controllers.

sim: Add the notion of clock domains to all ClockedObjects

This patch adds the notion of source- and derived-clock domains to the
ClockedObjects. As such, all clock information is moved to the clock
domain, and the ClockedObjects are grouped into domains.

The clock domains are either source domains, with a specific clock
period, or derived domains that have a parent domain and a divider
(potentially chained). For piece of logic that runs at a derived clock
(a ratio of the clock its parent is running at) the necessary derived
clock domain is created from its corresponding parent clock
domain. For now, the derived clock domain only supports a divider,
thus ensuring a lower speed compared to its parent. Multiplier
functionality implies a PLL logic that has not been modelled yet
(create a separate clock instead).

The clock domains should be used as a mechanism to provide a
controllable clock source that affects clock for every clocked object
lying beneath it. The clock of the domain can (in a future patch) be
controlled by a handler responsible for dynamic frequency scaling of
the respective clock domains.

All the config scripts have been retro-fitted with clock domains. For
the System a default SrcClockDomain is created. For CPUs that run at a
different speed than the system, there is a seperate clock domain
created. This domain incorporates the CPU and the associated
caches. As before, Ruby runs under its own clock domain.

The clock period of all domains are pre-computed, such that no virtual
functions or multiplications are needed when calling
clockPeriod. Instead, the clock period is pre-computed when any
changes occur. For this to be possible, each clock domain tracks its
children.

config: Add a system clock command-line option

This patch adds a 'sys_clock' command-line option and use it to assign
clocks to the system during instantiation.

As part of this change, the default clock in the System class is
removed and whenever a system is instantiated a system clock value
must be set. A default value is provided for the command-line option.

The configs and tests are updated accordingly.

ruby: remove the functional copy of memory in se mode
This patch removes the functional copy of the memory that was maintained in
the se mode. Now ruby itself will provide the data.

Configs: Set the memtest clock to a reasonable value

This patch changes the memtest clock from 1THz (the default) to 2GHz,
similar to the CPUs in the other regressions. This is useful as the
caches will adopt the same clock as the CPU. The bus clock rate is
scaled accordingly, and the L1-L2 bus is kept at the CPU clock while
the memory bus is at half that frequency.

A separate patch updates the affected stats.

Mem: Make SimpleMemory single ported

This patch changes the simple memory to have a single slave port
rather than a vector port. The simple memory makes no attempts at
modelling the contention between multiple ports, and any such
multiplexing and demultiplexing could be done in a bus (or crossbar)
outside the memory controller. This scenario also matches with the
ongoing work on a SimpleDRAM model, which will be a single-ported
single-channel controller that can be used in conjunction with a bus
(or crossbar) to create a multi-port multi-channel controller.

There are only very few regressions that make use of the vector port,
and these are all for functional accesses only. To facilitate these
cases, memtest and memtest-ruby have been updated to also have a
"functional" bus to perform the (de)multiplexing of the functional
memory accesses.

regress: ruby stat additions and config changes

MEM: Enable multiple distributed generalized memories

This patch removes the assumption on having on single instance of
PhysicalMemory, and enables a distributed memory where the individual
memories in the system are each responsible for a single contiguous
address range.

All memories inherit from an AbstractMemory that encompasses the basic
behaviuor of a random access memory, and provides untimed access
methods. What was previously called PhysicalMemory is now
SimpleMemory, and a subclass of AbstractMemory. All future types of
memory controllers should inherit from AbstractMemory.

To enable e.g. the atomic CPU and RubyPort to access the now
distributed memory, the system has a wrapper class, called
PhysicalMemory that is aware of all the memories in the system and
their associated address ranges. This class thus acts as an
infinitely-fast bus and performs address decoding for these "shortcut"
accesses. Each memory can specify that it should not be part of the
global address map (used e.g. by the functional memories by some
testers). Moreover, each memory can be configured to be reported to
the OS configuration table, useful for populating ATAG structures, and
any potential ACPI tables.

Checkpointing support currently assumes that all memories have the
same size and organisation when creating and resuming from the
checkpoint. A future patch will enable a more flexible
re-organisation.

Config: Change the way options are added
I am not too happy with the way options are added in files se.py and fs.py
currently. This patch moves all the options to the file Options.py, functions
from which are called when required.

MEM: Introduce the master/slave port roles in the Python classes

This patch classifies all ports in Python as either Master or Slave
and enforces a binding of master to slave. Conceptually, a master (such
as a CPU or DMA port) issues requests, and receives responses, and
conversely, a slave (such as a memory or a PIO device) receives
requests and sends back responses. Currently there is no
differentiation between coherent and non-coherent masters and slaves.

The classification as master/slave also involves splitting the dual
role port of the bus into a master and slave port and updating all the
system assembly scripts to use the appropriate port. Similarly, the
interrupt devices have to have their int_port split into a master and
slave port. The intdev and its children have minimal changes to
facilitate the extra port.

Note that this patch does not enforce any port typing in the C++
world, it merely ensures that the Python objects have a notion of the
port roles and are connected in an appropriate manner. This check is
carried when two ports are connected, e.g. bus.master =
memory.port. The following patches will make use of the
classifications and specialise the C++ ports into masters and slaves.

Merge with main repository.

SE/FS: Make both SE and FS tests available all the time.

SE/FS: Make SE vs. FS mode a runtime parameter.

Ruby: Connect system port in Ruby network test

This patch moves the connection of the system port to create_system in
Ruby.py. Thereby it allows the failing Ruby test (and other Ruby
systems) to run again.

MEM: Add port proxies instead of non-structural ports

Port proxies are used to replace non-structural ports, and thus enable
all ports in the system to correspond to a structural entity. This has
the advantage of accessing memory through the normal memory subsystem
and thus allowing any constellation of distributed memories, address
maps, etc. Most accesses are done through the "system port" that is
used for loading binaries, debugging etc. For the entities that belong
to the CPU, e.g. threads and thread contexts, they wrap the CPU data
port in a port proxy.

The following replacements are made:
FunctionalPort > PortProxy
TranslatingPort > SETranslatingPortProxy
VirtualPort > FSTranslatingPortProxy

Ruby: Add support for functional accesses
This patch rpovides functional access support in Ruby. Currently only
the M5Port of RubyPort supports functional accesses. The support for
functional through the PioPort will be added as a separate patch.

config: tweak ruby configs to clean up hierarchy

Re-enabling implicit parenting (see previous patch) causes current
Ruby config scripts to create some strange hierarchies and generate
several warnings. This patch makes three general changes to address
these issues.

1. The order of object creation in the ruby config files makes the L1
caches children of the sequencer rather than the controller; these
config ciles are rewritten to assign the L1 caches to the
controller first.

2. The assignment of the sequencer list to system.ruby.cpu_ruby_ports
causes the sequencers to be children of system.ruby, generating
warnings because they are already parented to their respective
controllers. Changing this attribute to _cpu_ruby_ports fixes this
because the leading underscore means this is now treated as a plain
Python attribute rather than a child assignment. As a result, the
configuration hierarchy changes such that, e.g.,
system.ruby.cpu_ruby_ports0 becomes system.l1_cntrl0.sequencer.

3. In the topology classes, the routers become children of some random
internal link node rather than direct children of the topology.
The topology classes are rewritten to assign the routers to the
topology object first.

memtest: due to contention increase, increased deadlock threshold

regress: Regression tester updates

Regression tester updates required by the following patches:

brad/moved_python_protocol_files: config: moved python protocol config files
brad/ruby_options_movement: config: reorganized how ruby specifies command-line options
brad/config_token_bcast: ruby: added token broadcast config params to cmd options
brad/topology_name: config: Added the topology description to m5 config.ini
brad/ruby_system_names: config: Improve ruby simobject names
brad/consolidated_protocol_stats: slicc: Consolidated the protocol stats printing
brad/ruby_request_type_ostream_fix: ruby: Added ruby_request_type ostream def to libruby.hh
brad/memtest_dma_extension: memtest: Memtester support for DMA
brad/token_dma_lockdown_fix: MOESI_CMP_token: Fixed dma persistent lockdown bugs
brad/profile_generic_mach_type: ruby: Reincarnated the responding machine profiling
brad/network_msg_consolidated_stats: ruby: Added consolidated network msg stats
brad/bcast_msg_profiling: ruby: Added bcast msg profiling to hammer and token
brad/l2cache_profiling_fix: ruby: Fixed L2 cache miss profiling
brad/llsc_ruby_m5_fix: ruby: fix ruby llsc support to sync sc outcomes
brad/ruby_latency_fixes: ruby: Reduced ruby latencies
brad/hammer_l2_cache_latency: ruby: Updated MOESI_hammer L2 latency behavior
brad/deterministic_resurrection: ruby: Resurrected Ruby's deterministic tests
brad/token_dma_fixes: ruby: MOESI_CMP_token dma fixes
brad/ruby_cmd_options: config: added cmd options to control ruby debug
brad/token_owner_fixes: ruby: fixed token bugs associated with owner token counts
brad/ruby_remove_try_except: ruby: Improved try except blocks in ruby creation
brad/ruby_port_callback_fix: ruby: Fixed RubyPort sendTiming callbacks
brad/interrupt_drain_fix: devices: Fixed periodic interrupts to work with draining
brad/llsc_trace_profile: ruby: Added SC fail indication to trace profiling
brad/no_migrate_atomic: ruby: Disable migratory sharing for token and hammer
brad/ruby_start_time_fix: ruby: Reset ruby stats in RubySystem unserialize
brad/numa_bit_select_fix: ruby: fixed DirectoryMemory's numa_high_bit configuration
brad/hammer_probe_filter: ruby: added probe filter support to hammer
brad/miss_latency_detail_profile: MOESI_hammer: break down miss latency stalled cycles
brad/recycle_latency_fix: ruby: Recycle latency fix for hammer
brad/stall_and_wait: ruby: Stall and wait input messages instead of recycling
brad/rubytest_request_flag_fix: ruby: Fixed minor bug in ruby test for setting the request type
brad/hammer_merge_gets: ruby: Added merge GETS optimization to hammer
brad/regress_updates: regress: Regression tester updates

ruby: Regression updates for new ruby config locations

m5: Regression Tester Update

This patch includes the necessary regression updates to test the new ruby
configuration system. The patch includes support for multiple ruby protocols
and adds the ruby random tester. The patch removes atomic mode test for
ruby since ruby does not support atomic mode acceses. These tests can be
added back in when ruby supports atomic mode for real.

ruby: memtest-ruby updated to the new config system

merge

ruby: cleaned up unified MESI/MOESI configuration

m5: refreshed the ruby memtest regression stats

ruby: Fix RubyMemory to work with the newer ruby.

ruby: Set up Ruby regression tests.