scons: Switch from the print statement to the print function.

Starting with version 3, scons imposes using the print function instead
of the print statement in code it processes. To get things building
again, this change moves all python code within gem5 to use the
function version. Another change by another author separately made this
same change to the site_tools and site_init.py files.

Change-Id: I2de7dc3b1be756baad6f60574c47c8b7e80ea3b0
Reviewed-on: https://gem5-review.googlesource.com/8761
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>

scons: Move Transform and termcap functionality into their own files.

Change-Id: Ica08e93f3873a7eafd02fe7d44c3bdbf0ce7f6b7
Reviewed-on: https://gem5-review.googlesource.com/5565
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>

tests: Split test results into running and verification

The test base class already assumes that test cases consists of a run
stage and a verification stage. Reflect this in the results class to
make it possible to detect cases where a run was successful, but
didn't verify.

Change-Id: I31ef393e496671221c5408aca41649cd8dda74ca
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>

scons: Use the new test framework from scons

Rewrite the SCons script responsible for launching tests to use the
new test framework. The biggest visible change after this changeset is
that SCons no longer produces a "status" file in test build
directories. Instead, it creates a status.pickle file. That file can
be inspected using the new tests.py script. Another visible change is
that timed out tests are now reported as failed rather than a using a
separate summary message.

Since the pickle file will remain in the build directory after running
tests, it's now possible to convert the test results into other
formats. For example:

./tests/tests.py show --format junit -o junit.xml \
`find build/ -name status.pickle`

To better facilitate running outside of scons, there is now a set of
targets that generate test listings that can be used by the test
script. There are currently three targets, quick.list, long.list, and
all.list. For example:

scons build/ARM/tests/opt/all.list
for F in `cat build/ARM/tests/opt/all.list`; do
./tests/tests.py run build/ARM/gem5.opt $F
done

Change-Id: I2c0f8ca0080d7af737362e198eda4cb3a72e6c36
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>

tests: Add a basic memcheck regression

This patch adds a simple regression that calls the existing
memcheck.py script.

gpu-compute: AMD's baseline GPU model

tests: Update SMT tests to correctly configure CPUs

The 01.hello-2T-smt test case for the O3 CPU didn't correctly setup
the number of threads before creating interrupt controllers, which
confused the constructor in BaseCPU. This changeset adds SMT support
to the test configuration infrastructure.

tests: Add tests for the Learning gem5 scripts

These tests will ensure that Learning gem5 scripts are always up to date with
the changes in the mainline of gem5.

Committed by: Nilay Vaish <nilay@cs.wisc.edu>

test, arm: Add scripts to test checkpoints

Add a set of scripts to automatically test checkpointing in the
regression framework. The checkpointing tests are similar to the
switcheroo tests, but instead of switching between CPUs, they
checkpoint the system and restore from the checkpoint again. This is
done at regular intervals, typically while booting Linux.

The implementation is fairly straight forward, with the exception that
we have to work around gem5's inability to restore from a checkpoint
after a system has been instantiated. We work around this by forking
off child processes that does the actual simulation and never
instantiate a system in the parent process unless a maximum checkpoint
count is reached (in which case we just simulate the system to
completion in the parent).

Checkpoint testing is currently only enabled 32- and 64-bit ARM
systems using atomic CPUs.

Note: An unfortunate side-effect of forking is that every new process
will overwrite the stats and terminal output from the previous
process. This means that the output directory only contains data from
the last checkpoint.

tests: Bump timeout to 5 hours

Align with observed run-times just above 4 hours for some hosts.

tests: Run regression timeout as foreground

Allow the user to send signals such as Ctrl C to the gem5 runs. Note
that this assumes coreutils >= 8.13, which aligns with Ubuntu 12.04
and RHE6.

tests: Remove deprecated InOrderCPU tests

This patch removes the three MIPS and SPARC regressions that use the
deprecated InOrderCPU.

This is the first step in completely removing the code from the tree,
avoiding confusion, and focusing all development efforts on the
MinorCPU. Brave new world.

arm, tests: Update config files to more recent kernels and create 64-bit regressions.

This changes the default ARM system to a Versatile Express-like system that supports
2GB of memory and PCI devices and updates the default kernels/file-systems for
AArch64 ARM systems (64-bit) to support up to 32GB of memory and PCI devices. Some
platforms that are no longer supported have been pruned from the configuration files.

In addition a set of 64-bit ARM regressions have been added to the regression system.

tests: Add a memtest version using the ideal SnoopFilter

This patch adds a basic regression test for the snoop filter.

tests: automatically kill regressions that take too long

When GNU coreutils 'timeout' is available, limit each regression
simulation to 4 hours.

alpha: Stop using 'inorder' and rely entirely on 'minor'

This patch avoids building the 'inorder' CPU model for any permutation
of ALPHA, and also removes the ALPHA regressions using the 'inorder'
CPU. The 'minor' CPU is already providing a broader test coverage.

cpu: Minor CPU add regression tests for ARM and ALPHA

This patch adds regression tests results and test harnesses
for the Minor CPU on ARM and ALPHA.

tests: Reflect name change in DRAM tests

This patch reflects the recent name change in the DRAM TrafficGen
tests and also tidies up the test directory.

arch: teach ISA parser how to split code across files

This patch encompasses several interrelated and interdependent changes
to the ISA generation step. The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.

The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units. I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.

Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner. It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type. Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.

Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths. In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.

Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.

scons: fix minor update-ref bug in regressions

In the unusual case that regressions are run with --update-ref
when there is no existing regression output, scons gets
confused because it depends on stats.txt to trigger the
update, but it has no indication that running the test will
generate the stats.txt file. (In the typical case where
stats.txt already exists, scons doesn't care about where
it came from.)

It's easy to fix this just by adding the stats.txt file
to the target list for the test action.

scons: Identify runs that fail and runs with stats differences

This patch changes the regression script such that it is possible to
identify the runs that fail with an exit code, and those that finish
with stats differences. The ones that truly fail are reported as
FAILED, and those that finish with changed stats as CHANGED.

The yellow colour has been reclaimed from the skipped regressions and
is now used for the changed ones. With no obvious good option left the
skipped ones are now in cyan.

While I was editing the script I also bumped any occurence of M5 to
gem5.

x86: regressions: add switcher full test

tests: Add CPU switching tests

This changeset adds a set of tests that stress the CPU switching
code. It adds the following test configurations:

* tsunami-switcheroo-full -- Alpha system (atomic, timing, O3)
* realview-switcheroo-atomic -- ARM system (atomic<->atomic)
* realview-switcheroo-timing -- ARM system (timing<->timing)
* realview-switcheroo-o3 -- ARM system (O3<->O3)
* realview-switcheroo-full -- ARM system (atomic, timing, O3)

Reference data is provided for the 10.linux-boot test case. All of the
tests trigger a CPU switch once per millisecond during the boot
process.

The in-order CPU model was not included in any of the tests as it does
not support CPU handover.

tests: Update the ignore regexps to reflect the M5->gem5 name change

tests: Add support for skipping tests, skip EIO tests if not enabled

The EIO tests depend on the EIO support from the "encumbered"
repository, which means that they are not normally built with
gem5. This causes all EIO related tests to fail, which is both
annoying and confusing. This patch addresses this by adding support
for skipping tests if certain conditions (e.g., the presence of a
SimObject) can not be met. It introduces the following Python
functions that can be called from within a test case:

* skip_test -- Skip a test and optionally print why the test was
skipped.

* has_sim_object -- Test if a SimObject exists.

* require_sim_object -- Test if a SimObject exists and skip, or
optionally fail, the test if not.

Additionally, this patch updates the EIO tests to check for the
presence of EioProcess.

SimpleDRAM: A basic SimpleDRAM regression

TrafficGen: Add a basic traffic generator regression

This patch adds a basic regression for the traffic generator. The
regression also serves as an example of the file formats used. More
complex regressions that make use of a DRAM controller model will
follow shortly.

Regression: Add ANSI colours to highlight test status

This patch adds a very basic pretty-printing of the test status
(passed or failed) to highlight failing tests even more: green for
passed, and red for failed. The printing only uses ANSI it the target
output is a tty and supports ANSI colours. Hence, any regression
scripts that are outputting to files or sending e-mails etc should
still be fine.

CheckerCPU: Make some basic regression tests for CheckerCPU

Adds regression tests for the CheckerCPU. ARM ISA support
only at this point.

SE/FS: Make both SE and FS tests available all the time.

tests: fix spurious scons "Error 1" messages

Turns out these are due to diff reporting that files
acutally differed via a non-zero exit code.

ARM: Add some MP regressions and clean up the disk images and kernels a bit

SCons,tests: Tell scons about pc-o3-timing regressions.

BuildEnv: Eliminate RUBY as build environment variable
This patch replaces RUBY with PROTOCOL in all the SConscript files as
the environment variable that decides whether or not certain components
of the simulator are compiled.

Scons: Drop RUBY as compile time option.
This patch drops RUBY as a compile time option. Instead the PROTOCOL option
is used to figure out whether or not to build Ruby. If the specified protocol
is 'None', then Ruby is not compiled.

inorder: make InOrder CPU FS compilable/visible
make syscall a SE mode only functionality
copy over basic FS functions (hwrei) to make FS compile

ARM: Update stats for the previous changes and add ARM_FS/O3 regression.

SCons: Turn some scons variables into command line options.

X86: Add scripts to support X86 FS configurations in the regressions.

ARM: Add full-system regressions

tests: print if output files match
Add '-s' flag to diff command generating outdiff
file so we have positive confirmation when
outputs match.

m5: Regression Tester Update

This patch includes the necessary regression updates to test the new ruby
configuration system. The patch includes support for multiple ruby protocols
and adds the ruby random tester. The patch removes atomic mode test for
ruby since ruby does not support atomic mode acceses. These tests can be
added back in when ruby supports atomic mode for real.

Add ability to skip tests by adding 'skip' file to test dir,
and skip simple-timing-mp-ruby test for now (until we fix ruby atomics).

inorder-regress: add hello world

ruby: add RUBY sticky option that must be set to add ruby to the build
Default is false

ruby: Set up Ruby regression tests.

Set up m5threads tests on classic (non-ruby) memory system.
Just one test (40.m5threads-test-atomic) is set up for now.
These tests require that the m5threads SPARC binaries are present
in /dist or in test-progs.

alpha: get rid of all turbolaser remnants

tests: use env.Execute instead of Execute to pick up env vars.

Minor tweak to regression exit status message.

Fix up regression execution to better handle tests that end abnormally.
E.g., mark aborts due to assertion failures as failed tests,
but those that get killed by the user as needing to be rerun, etc.

output: Change default output directory and files and update tests.

new mp eio test

Regression: Add single and dual boot O3 regressions. They both take about 8 minutes to complete.

Minor fix for test/SConscript... forgot to 'qref' before 'qdel', argh.

Make test/SConscript use new redirection options.

tests: replace all dest ref files on upgrade (if possible).
Originally we were copying all source files in, but this caused
problems when (large) inputs were copied along with outputs.
Then we switched to just copying the standard files (m5stats.txt,
etc.) but that was missing things like the *.console files.
This fix should catch all the non-standard files too as long as
they are copied in manually once when the test is set up.
Also get a lot nicer about warning when files are ignored,
and warn when expected files are missing.
Those new Python sets sure are handy.

tests: config.out no longer exists, eliminate ref copy.

style: Check/Fix whitespace on SCons files

add a sparc fs regression

src/dev/sparc/iob.cc:
don't warn on cpu restart/idle/halt stuff
tests/SConscript:
add sparc target in test Sconscript
util/regress:
Add SPARC_FS target in regress

Add short memtest run to quick regressions.
Caveats:
- Even though memtest is ISA-independent, it will only
run for the Alpha builds, since there's no way to specify
ISA-independent reference files and I didn't want to commit
3 copies since I'm not sure we want to run it for all the
different ISAs anyway.
- Reference outputs were generated on my laptop,
so performance numbers will be low compared to zizzer.

Only update stderr, stdout, m5stats.txt, and config.* on update_ref,
since we don't know which of the other files are outputs and which
are inputs.

add a simple netperf-stream test to the long tests.

tests/SConscript:
add a new configuration for two-system tests (atomic simple only)

Add o3-timing configuration for ALPHA_SE "Hello world" tests.

build_opts/ALPHA_SE:
Add O3CPU to default CPU model list.
tests/SConscript:
Add o3-timing configuration.

SConscript:
Fix BATCH_CMD bug.

tests/SConscript:
Fix BATCH_CMD bug.

More regression updates.
Get rid of caches in simple-timing config for now.

tests/SConscript:
another line for diff to ignore
tests/configs/simple-timing.py:
turn off caches for now
tests/quick/00.hello/ref/mips/linux/simple-atomic/m5stats.txt:
tests/quick/00.hello/ref/mips/linux/simple-atomic/stdout:
tests/quick/00.hello/ref/sparc/linux/simple-atomic/m5stats.txt:
tests/quick/00.hello/ref/sparc/linux/simple-atomic/stdout:
update for inst/tick rate (old one was debug?)
tests/quick/20.eio-short/ref/alpha/eio/simple-timing/config.ini:
tests/quick/20.eio-short/ref/alpha/eio/simple-timing/config.out:
tests/quick/20.eio-short/ref/alpha/eio/simple-timing/m5stats.txt:
tests/quick/20.eio-short/ref/alpha/eio/simple-timing/stderr:
tests/quick/20.eio-short/ref/alpha/eio/simple-timing/stdout:
works now (no caches)

Minor regression fixes.

src/python/m5/objects/BaseCPU.py:
bug fix
tests/SConscript:
fix up diff ignore strings to reflect changes
in m5 output

Finish test clean-up & reorg.

configs/common/FSConfig.py:
Add default Machine() param
configs/example/fs.py:
configs/example/se.py:
make it work again
src/python/m5/objects/BaseCPU.py:
Make mem PhysicalMemory so that a Parent.any proxy works well
src/sim/process.cc:
Increase default stack size so we don't get an
'increasing stack' message on 'hello world'
tests/SConscript:
Add full list of current configs.
tests/configs/simple-atomic.py:
tests/configs/simple-timing.py:
don't need SEConfig anymore
tests/quick/00.hello/test.py:
tests/quick/20.eio-short/test.py:
fix
tests/run.py:
move configs to separate dir

Halfway through setting up new test structure... committing so
O can move to my laptop.

tests/SConscript:
Start to simplify.

Last minute check in. Very few functional changes other than some minor config updates. Also include some recently generated stats.

SConstruct:
Make test CPUs option non-sticky.
configs/common/FSConfig.py:
Be sure to set the memory mode.
configs/test/fs.py:
Wrong string.
tests/SConscript:
Only test valid CPUs that have been compiled in.
tests/test1/ref/alpha/atomic/config.ini:
tests/test1/ref/alpha/atomic/config.out:
tests/test1/ref/alpha/atomic/m5stats.txt:
tests/test1/ref/alpha/atomic/stdout:
tests/test1/ref/alpha/detailed/config.ini:
tests/test1/ref/alpha/detailed/config.out:
tests/test1/ref/alpha/detailed/m5stats.txt:
tests/test1/ref/alpha/detailed/stdout:
tests/test1/ref/alpha/timing/config.ini:
tests/test1/ref/alpha/timing/config.out:
tests/test1/ref/alpha/timing/m5stats.txt:
tests/test1/ref/alpha/timing/stdout:
Update output.

Minor functionality updates.

SConstruct:
Include an option to specify the CPUs being tested.
src/cpu/SConscript:
Checker isn't SMT right now, so don't do SMT tests with the O3CPU if we're using the checker.
src/python/m5/objects/O3CPU.py:
Include default options. Unfortunately FullO3Config.py is still needed because it specifies which FUPool is being used.
tests/SConscript:
Several minor updates (sorry for one commit). Updated the copyright and fixed some m5 style issues. Also added the ability to specify which CPUs to run the tests on.

Put regression tests back into m5. They are located in the "tests" directory. The directory output and reference outputs have changed slightly. Now the directory is ALPHA_SE/test/<test>/<cpu_model>/, and for the reference stats <test>/ref/<arch>/<cpu_model>

Right now only non-SMT SE regression tests have been added back in. The rest are pending getting SMT working, and consolidating the FS configuration files.

Eventually support for different OSs can be added so you can specify which versions of the binary you want to run from one config file.

Note: mp-test1 doesn't have any reference stats because MP mode doesn't currently work. The test itself should probably work once the code is fixed.

SConstruct:
Updates to allow for regression tests to work via the command line "scons build/ALPHA_SE/test/debug/quick" and such once again.
src/cpu/SConscript:
Keep a list of SMT supporting CPUs so that the regression tests can easily specify which CPUs to use if they are SMT only.