stats: Add M5_VAR_USED to variable

Fixes compile error for gem5.fast due to unused variable.

Change-Id: Ib1664b100e95a741174610f8cdbdebaff9635f66
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20759
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

stats: Fix incorrect name conflict panic with grouped stats

Info::setName() performs a sanity check to ensure that the same stat
name isn't used twice. This doesn't work for new-style stats with a
parent group since the name is only unique within the group. Disable
the check for new-style stats since these usually use names generated
from member variable names.

Change-Id: I590abe6040407c6a4fe582c0782a418165ff5588
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20760
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Tested-by: kokoro <noreply+kokoro@google.com>

stats: Create HDF5 stat files relative to simout

When using the HDF5 stats backend, we currently pass the filename from
the command line straight to the HDF5 library. This behaviour is
different from the expected behaviour when using plain text stat files
where relative paths are resolved relative to simout. This change adds
support for resolving hdf5 paths relative to simout.

Change-Id: I09cb0b7468735e697125eda7f04c5a85725ca8d0
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20508
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>

stats: Catch exceptions by const reference

Catching exceptions by value causes -Werror=catch-value= to be
triggered during a build process. This change replaces the values
with const references in the catch blocks.

Change-Id: Iddabb595c5916d8120dcdb08e2c2f36ff21c3c9b
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/20528
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

stats: Add beta support for HDF5 stat dumps

This changeset add support for stat dumps in the HDF5 file
format. HDF5 is a binary data format that represents data in a
file-system-like balanced tree. It has native support for
N-dimensional arrays and binary data (e.g., frame buffers).

It has the following benefits over traditional text stat files:

* Efficient storage of time series (multiple stat dumps)

* Fast lookup of stats

* Plenty of existing tooling (e.g., Python libraries and graphical
viewers)

* File format can be used to store frame buffers together with
normal stats.

Drawbacks:

* Large startup cost (single stat dump larger than text equivalent)

* Stat dumps are slower than text

Known limitations:

* Distributions and histograms aren't supported.

HDF5 stat output can be enabled using the 'h5' URL scheme when
overriding the stat file name on gem5's command line. The following
parameters are supported:

* chunking (unsigned): Number of time steps to pre-allocate
(default: 10)

* desc (bool): Output stat descriptions (default: True)

* formulas (bool): Output derived stats (default: True)

Example gem5 command line:

./build/ARM/gem5.opt \
--stats-file="h5://stats.h5?desc=False;formulas=False" \
configs/example/fs.py

Example Python stat consumer that computes IPC:
import h5py

f = h5py.File('stats.h5', 'r')
group = f['/system/cpu']
for i, c in zip(group['committedInsts'], group['numCycles']):
print i, c, i / c

Change-Id: I351c6cbff2fb7bef9012f47876ba227ed288975b
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/8121
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>

stats: Add support for hierarchical stats

This change makes the stat system aware of the hierarchical nature of
stats. The aim is to achieve the following goals:

* Make the SimObject hierarchy explicit in the stat system (i.e.,
get rid of name() + ".foo"). This makes stat naming less fragile
and makes it possible to implement hierarchical formats like
XML/HDF5/JSON in a clean way.

* Make it more convenient to split stats into a separate
struct/class that can be bound to a SimObject. This makes the
namespace cleaner and makes stat accesses a bit more obvious.

* Make it possible to build groups of stats in C++ that can be used
in subcomponents in a SimObject (similar to what we do for
checkpoint sections). This makes it easier to structure large
components.

* Enable partial stat dumps. Some of our internal users have been
asking for this since a full stat dump can be large.

* Enable better stat access from Python.

This changeset implements solves the first three points by introducing
a class (Stats::Group) that owns statistics belonging to the same
object. SimObjects inherit from Stats::Group since they typically have
statistics.

New-style statistics need to be associated with a parent group at
instantiation time. Instantiation typically sets the name and the
description, other parameters need to be set by overriding
Group::regStats() just like with legacy stats. Simple objects with
scalar stats can typically avoid implementing regStats() altogether
since the stat name and description are both specified in the
constructor.

For convenience reasons, statistics groups can be merged into other
groups. This means that a SimObject can create a stat struct that
inherits from Stats::Group and merge it into the parent group
(SimObject). This can make the code cleaner since statistics tracking
gets grouped into a single object.

Stat visitors have a new API to expose the group structure. The
Output::beginGroup(name) method is called at the beginning of a group
and the Output::endGroup() method is called when all stats, and
sub-groups, have been visited. Flat formats (e.g., the text format)
typically need to maintain a stack to track the full path to a stat.

Legacy, flat, statistics are still supported after applying this
change. These stats don't belong to any group and stat visitors will
not see a Output::beginGroup(name) call before their corresponding
Output::visit() methods are called.

Change-Id: I9025d61dfadeabcc8ecf30813ab2060def455648
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19368
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

misc: Rename misc.(hh|cc) to logging.(hh|cc)

These files aren't a collection of miscellaneous stuff, they're the
definition of the Logger interface, and a few utility macros for
calling into that interface (panic, warn, etc.).

Change-Id: I84267ac3f45896a83c0ef027f8f19c5e9a5667d1
Reviewed-on: https://gem5-review.googlesource.com/6226
Reviewed-by: Brandon Potter <Brandon.Potter@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>

style: [patch 1/22] use /r/3648/ to reorganize includes

base: Add total() to Vector2D stat

This patch adds a total() function to the Vector2D
stat type. Similar to other stats such as Scalar or
Vector it is useful to be able to read the total for
a given stat.

base: Add support for changing output directories

This changeset adds support for changing the simulator output
directory. This can be useful when the simulation goes through several
stages (e.g., a warming phase, a simulation phase, and a verification
phase) since it allows the output from each stage to be located in a
different directory. Relocation is done by calling core.setOutputDir()
from Python or simout.setOutputDirectory() from C++.

This change affects several parts of the design of the gem5's output
subsystem. First, files returned by an OutputDirectory instance (e.g.,
simout) are of the type OutputStream instead of a std::ostream. This
allows us to do some more book keeping and control re-opening of files
when the output directory is changed. Second, new subdirectories are
OutputDirectory instances, which should be used to create files in
that sub-directory.

Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

base: Clean up redundant string functions and use C++11

This patch does a bit of housekeeping on the string helper functions
and relies on the C++11 standard library where possible. It also does
away with our custom string hash as an implementation is already part
of the standard library.

stats: Fix flow-control bug in Vector2D printing

stats: add function for adding two histograms
This patch adds a function to the HistStor class for adding two histograms.
This functionality is required for Ruby. It also adds support for printing
histograms in a single line.

base: Fix VectorPrint initialisation

This patch changes how the initialisation of the VectorPrint struct is
done so that gcc 4.4 is happy again.

stats: Fix issue when printing 2D vectors

This patch addresses an issue with the text-based stats output which
resulted in Vector2D stats being printed without subnames in the event
that one of the dimensions was of length 1.

This patch also fixes the total printing for the 2D vector. Previously
totals were printed without explicitly stating that a total was being
printed. This has been rectified in this patch.

stats: Remove printing of SparseHist total

This patch removes the printing of the SparseHist total in the
stats.txt output file. This has been removed as a sparse histogram has
no total, and therefore this was printing out the value of a
non-local, unrelated variable.

stats: allow printing vectors on a single line
This patch adds a new flag to specify if the data values for a given vector
should be printed in one line in the stats.txt file. The default behavior
will be to print the data in multiple lines. It makes changes to print
functions to enforce this behavior.

scons: Add warning for missing declarations

This patch enables warnings for missing declarations. To avoid issues
with SWIG-generated code, the warning is only applied to non-SWIG
code.

scons: Enforce gcc >= 4.4 or clang >= 2.9 and c++0x support

This patch checks that the compiler in use is either gcc >= 4.4 or
clang >= 2.9. and enables building with --std=c++0x in all cases. As a
consequence, we can tidy up the hashmap and always have static_assert
available. If anyone wants to use alternative compilers, icc for
example supports c++0x to a similar level and could be added if
needed.

This patch opens up for a more elaborate use of c++0x features that
are present in gcc 4.4 and clang 2.9, e.g. auto typed variables,
variadic templates, rvalues and move semantics, and strongly typed
enums. There will be no going back on this one...

stats: use nan instead of no_value

clang/gcc: Fix compilation issues with clang 3.0 and gcc 4.6

This patch addresses a number of minor issues that cause problems when
compiling with clang >= 3.0 and gcc >= 4.6. Most importantly, it
avoids using the deprecated ext/hash_map and instead uses
unordered_map (and similarly so for the hash_set). To make use of the
new STL containers, g++ and clang has to be invoked with "-std=c++0x",
and this is now added for all gcc versions >= 4.6, and for clang >=
3.0. For gcc >= 4.3 and <= 4.5 and clang <= 3.0 we use the tr1
unordered_map to avoid the deprecation warning.

The addition of c++0x in turn causes a few problems, as the
compiler is more stringent and adds a number of new warnings. Below,
the most important issues are enumerated:

1) the use of namespaces is more strict, e.g. for isnan, and all
headers opening the entire namespace std are now fixed.

2) another other issue caused by the more stringent compiler is the
narrowing of the embedded python, which used to be a char array,
and is now unsigned char since there were values larger than 128.

3) a particularly odd issue that arose with the new c++0x behaviour is
found in range.hh, where the operator< causes gcc to complain about
the template type parsing (the "<" is interpreted as the beginning
of a template argument), and the problem seems to be related to the
begin/end members introduced for the range-type iteration, which is
a new feature in c++11.

As a minor update, this patch also fixes the build flags for the clang
debug target that used to be shared with gcc and incorrectly use
"-ggdb".

stats: fix Vector2d to display stats correctly when y_subname is not specified.

Vector2d stats with no y_subname were not displayed as the VectorPrint subname was not initialized correctly to reflect the empty field.

sim: Enable sampling of run-time for code-sections marked using pseudo insts.

This patch adds a mechanism to collect run time samples for specific portions
of a benchmark, using work_begin and work_end pseudo instructions.It also enhances
the histogram stat to report geometric mean.

Output: Add hierarchical output support and cleanup existing codebase.

Stats: Add a sparse histogram stat object.

gcc 4.0: Add some virtual destructors to make gcc 4.0 happy.

stats: delete mysql support
we can add it back within python in some future changeset

stats: move code that loops over all stats into python

base: include types.hh in base/stats/mysql.hh
Due to certain changes made via changeset 8229, the compilation was failing
in certain cases. The compiler pointed to base/stats/mysql.hh for not naming
a certain types like uint64_t. To rectify this, base/types.hh is being
included in base/stats/mysql.hh.

stats: add user settable separator string for arrayed stats

Default is '::', so no visible change unless it is overridden

includes: fix up code after sorting

includes: sort all includes

stats: Add a histogram statistic type

Curtick: Fix mysql.cc build needing curTick.

Replace curTick global variable with accessor functions.
This step makes it easy to replace the accessor functions
(which still access a global variable) with ones that access
per-thread curTick values.

Make commenting on close namespace brackets consistent.

Ran all the source files through 'perl -pi' with this script:

s|\s*(};?\s*)?/\*\s*(end\s*)?namespace\s*(\S+)\s*\*/(\s*})?|} // namespace $3|;
s|\s*};?\s*//\s*(end\s*)?namespace\s*(\S+)\s*|} // namespace $2\n|;
s|\s*};?\s*//\s*(\S+)\s*namespace\s*|} // namespace $1\n|;

Also did a little manual editing on some of the arch/*/isa_traits.hh files
and src/SConscript.

stats: unify the two stats distribution type better

stats: cleanup a few small problems in stats

stats: rename print to display in the mysql code too...sorry

stats: rename print to display so it work in python

stats: get rid of the never-really-used event stuff

Stats: fix dist stat and enable VectorDistStat

compile: don't #include unnecessary stuff
Time from base/time.hh has a name clash with Time from Ruby's
TypeDefines.hh. Eventually Ruby's Time should go away, so instead of
fixing this properly just try to avoid the clash.

build: need to include cstdio

includes: sort includes again

types: Move stuff for global types into src/base/types.hh

stats: fancy is a bad name

stats: clean up the code for printing stats

stats: remove a few compat leftovers

stats: forgot an include for the mysql stuff

scons: add include guards to info.hh

stats: Move flags into info.hh and use base/flags.hh to manage the flags

stats: Shuffle around info stuff so it can be accessed separately

stats: Rename the info classes to hopefully make things a bit clearer
FooInfoBase became FooInfo
FooInfo became FooInfoProxy

stats: remove simplescalar compatibility for printing

stats: fix initialization bug in distribution text output

Merge ARM into the head. ARM will compile but may not actually work.

stats: cleanup text output stuff and fix mysql output

stats: create an enable phase, and a prepare phase.
Enable more or less takes the place of check, but also allows stats to
do some other configuration. Prepare moves all of the code that readies
a stat for dumping into a separate function in preparation for supporting
serialization of certain pieces of statistics data.
While we're at it, clean up the visitor code and some of the python code.

stats: stick the distribution's fancy parameter into the parameters structure.

stats: reorganize how parameters are stored and accessed.

stats: move the limits stuff into the types.hh file

stats: get rid of the convoluted 'database' code.
Just use the stuff directly and things ought to be more clear

stats: Try to make the names of things more intuitive.
Basically, this means renaming several things called data to info, which
is information about the statistics. Things that are named data now are
actual data stored for the statistic.

stats: fix text printout for distributions

stats: use properly signed types for looping and comparison

style: Bring statistics code in line with the proper style.

Output: Verify output files are open after opening them.

gcc: Add extra parens to quell warnings.
Even though we're not incorrect about operator precedence, let's add
some parens in some particularly confusing places to placate GCC 4.3
so that we don't have to turn the warning off. Agreed that this is a
bit of a pain for those users who get the order of operations correct,
but it is likely to prevent bugs in certain cases.

style: Remove non-leading tabs everywhere they shouldn't be. Developers should configure their editors to not insert tabs

Get rid of those pesky valgrind warnings, Conditional jump or move depends on uninitialised value(s), in the stats package

Do the default argument stuff in python

Fix majory brokenness in my previous MySQL commit, basically
this is just a shuffling around of code and fixes to make
stuff commit properly

Get rid of the Statistics and Statreset ParamContexts, and
expose all of the relevant functionality to python. Clean
up the mysql code while we're at it.

Remove the event_ignore stuff since it was never really used

make our code a little more standards compliant
pretty close to compiling w/ suns compiler

briefly:
add dummy return after panic()/fatal()
split out flags by compiler vendor
include cstring and cmath where appropriate
use std namespace for string ops

SConstruct:
Add code to detect compiler and choose cflags based on detected compiler
Fix zlib check to work with suncc
src/SConscript:
split out flags by compiler vendor
src/arch/sparc/isa/decoder.isa:
use correct namespace for sqrt
src/arch/sparc/isa/formats/basic.isa:
add dummy return around panic
src/arch/sparc/isa/formats/integerop.isa:
use correct namespace for stringops
src/arch/sparc/isa/includes.isa:
include cstring and cmath where appropriate
src/arch/sparc/isa_traits.hh:
remove dangling comma
src/arch/sparc/system.cc:
dummy return to make sun cc front end happy
src/arch/sparc/tlb.cc:
src/base/compression/lzss_compression.cc:
use std namespace for string ops
src/arch/sparc/utility.hh:
no reason to say something is unsigned unsigned int
src/base/compression/null_compression.hh:
dummy returns to for suncc front end
src/base/cprintf.hh:
use standard variadic argument syntax instead of gnuc specefic renaming
src/base/hashmap.hh:
don't need to define hash for suncc
src/base/hostinfo.cc:
need stdio.h for sprintf
src/base/loader/object_file.cc:
munmap is in std namespace not null
src/base/misc.hh:
use M5 generic noreturn macros
use standard variadic macro __VA_ARGS__
src/base/pollevent.cc:
we need file.h for file flags
src/base/random.cc:
mess with include files to make suncc happy
src/base/remote_gdb.cc:
malloc memory for function instead of having a non-constant in an array size
src/base/statistics.hh:
use std namespace for floor
src/base/stats/text.cc:
include math.h for rint (cmath won't work)
src/base/time.cc:
use suncc version of ctime_r
src/base/time.hh:
change macro to work with both gcc and suncc
src/base/timebuf.hh:
include cstring from memset and use std::
src/base/trace.hh:
change variadic macros to be normal format
src/cpu/SConscript:
add dummy returns where appropriate
src/cpu/activity.cc:
include cstring for memset
src/cpu/exetrace.hh:
include cstring fro memcpy
src/cpu/simple/base.hh:
add dummy return for panic
src/dev/baddev.cc:
src/dev/pciconfigall.cc:
src/dev/platform.cc:
src/dev/sparc/t1000.cc:
add dummy return where appropriate
src/dev/ide_atareg.h:
make define work for both gnuc and suncc
src/dev/io_device.hh:
add dummy returns where approirate
src/dev/pcidev.hh:
src/mem/cache/cache_impl.hh:
src/mem/cache/miss/blocking_buffer.cc:
src/mem/cache/tags/lru.hh:
src/mem/cache/tags/split.hh:
src/mem/cache/tags/split_lifo.hh:
src/mem/cache/tags/split_lru.hh:
src/mem/dram.cc:
src/mem/packet.cc:
src/mem/port.cc:
include cstring for string ops
src/dev/sparc/mm_disk.cc:
add dummy return where appropriate
include cstring for string ops
src/mem/cache/miss/blocking_buffer.hh:
src/mem/port.hh:
Add dummy return where appropriate
src/mem/cache/tags/iic.cc:
cast hastSets to double for log() call
src/mem/physical.cc:
cast pmemAddr to char* for munmap
src/sim/byteswap.hh:
make define work for suncc and gnuc

fixes so that M5 will compile under solaris

SConstruct:
Add check to see if we need to include libsocket
src/arch/sparc/floatregfile.cc:
src/arch/sparc/intregfile.cc:
use memset rather than bzero and include the appropriate headerfile
src/base/pollevent.cc:
If we're compling under solaris we need sys/file.h
src/base/random.cc:
src/base/random.hh:
solaris doesn't have random(), so use rint with the correct rounding mode
if we're compiling on solaris
src/base/stats/flags.hh:
u_int32_t??
src/base/time.hh:
grab the timersub() define from freebsd since it doesn't exist in solaris
src/cpu/inst_seq.hh:
we don't need to include stdint here
src/sim/byteswap.hh:
the method to detect endianness on Solaris is a little more complex...

fixes for gcc 4.1
Nate needs to fix sinic builder stuff
Gabe needs to verify my fixes to decoder.isa

OPT/DEBUG compiles for ALPHA_FS, ALPHA_SE, MIPS_SE, SPARC_SE with this changeset

README:
Fix the swig version in the readme
src/SConscript:
remove sinic until nate fixes the builder crap for it
src/arch/alpha/system.hh:
src/arch/mips/isa/includes.isa:
src/arch/sparc/isa/decoder.isa:
src/base/stats/visit.cc:
src/base/timebuf.hh:
src/dev/ide_disk.cc:
src/dev/sinic.cc:
src/mem/cache/miss/mshr.cc:
src/mem/cache/miss/mshr_queue.cc:
src/mem/packet.hh:
src/mem/request.hh:
src/sim/builder.hh:
src/sim/system.hh:
fixes for gcc 4.1

Merge iceaxe.:/Volumes/work/research/m5/head
into iceaxe.:/Volumes/work/research/m5/merge

src/cpu/simple/base.cc:
src/kern/kernel_stats.cc:
src/kern/kernel_stats.hh:
src/kern/system_events.cc:
src/kern/system_events.hh:
src/python/m5/objects/System.py:
src/sim/system.cc:
src/sim/system.hh:
hand merge

Updated Authors from bk prs info

New directory structure:
- simulator source now in 'src' subdirectory
- imported files from 'ext' repository
- support building in arbitrary places, including
outside of the source tree. See comment at top
of SConstruct file for more details.
Regression tests are temporarily disabled; that
syetem needs more extensive revisions.

SConstruct:
Update for new directory structure.
Modify to support build trees that are not subdirectories
of the source tree. See comment at top of file for
more details.
Regression tests are temporarily disabled.
src/arch/SConscript:
src/arch/isa_parser.py:
src/python/SConscript:
Update for new directory structure.