misc: updated shabang for python script

The default python on MacOS doesn’t have an alias to python2.
The official python version supported in gem5 is Python2.7.

This patch updates the shabang according to the version required in gem5.

Change-Id: I9533c0f7858b5b3cab0ef101be1ee5cd718105b0
Reviewed-on: https://gem5-review.googlesource.com/c/15375
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

tests,style: add RISC-V assembly tests

This patch adds a subset (rv64*) of RISC-V assembly tests. The original
riscv-test project can be found here:
https://github.com/riscv/riscv-tests. The riscv-test project is under the
BSD license (https://github.com/riscv/riscv-tests/blob/master/LICENSE)
and is maintained separately from gem5 project.

The tests have been slightly modified to work in gem5 SE mode:

(1) Removed a trap handler used in riscv-tests for bare-metal systems

(2) Instead of throwing an exception, the tests call the exit syscall
with
the exit code of
- '0' if SUCCESS
- Failed test case's number (non-zero) if FAILURE
The exit code can be captured after a simuation completes.

In addition to original RISC-V assembly tests, this patch adds several
assembly tests specifically for AMO, LR, SC and system calls. Those
tests target a multi-core system.

(1) rv64uamt: multi-threaded tests for A-extension instructions

(2) rv64samt: multi-threaded tests for clone and futex system calls

This patch also makes the style checker ignore RISC-V assembly test
directory. The assembly tests are maintained in an external project
that does not follow the gem5 coding conventions.

Please find more details in the README file included in this patch.

Change-Id: Id1015d9a2c6c7d0341fa8b81483289e5f0bfcec0
Reviewed-on: https://gem5-review.googlesource.com/6703
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>

style: Add shared gem5 headers to the style checker

Teach the style checker about common headers living in gem5/. These
should be included after any global library headers (e.g., C headers
or STL headers), but before the normal gem5 headers.

Change-Id: I322f841420e361c16314be8fa4cbd1e86d2bfa9f
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/4300
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

style: Treat PyBind headers as Python headers

Some PyBind11 headers need to include Python.h. This means that we need
to include PyBind11 headers before any standard library headers. Enforce
this by applying the Python.hh rules for anything in pybind11/.

Change-Id: Id175a4f613960a17f84f98b81bfd02806e905d5a
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/3120
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>

misc: Update #!env calls for python to explicit version

In some newer Linux distributions, env python default to Python 3.0. This
patch explicitly uses "python2" instead of just "python" for all scripts
that use #!

Reported-by: Sanchayan Maity <maitysanchayan@gmail.com>
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>

style: Force Python.h to be included before main header

Python's header files set various compiler macros (e.g.,
_XOPEN_SOURCE) unconditionally. This triggers preprocessor warnings
that end up being treated as errors. The Python integration manual [1]
strongly recommends that Python.h is included before any system
header. The style guide used to mandate that Python.h is included
first in any file that needs it. This requirement was changed to
always include a source file's main header first, which ended up
triggering these errors.

This change updates the style checker to always include Python.h
before the main header file.

[1] https://docs.python.org/2/extending/extending.html

Change-Id: Id6a4f7fc64a336a8fd26691a0ca682abeb1d1579
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Pierre-Yves Péneau <pierre-yves.peneau@lirmm.fr>

util: git pre-commit hook to check staged files

This patch updates the git-pre-commit hook to check the files as they
will be after the commit, instead of as they are currently, this way we
prevent the undesired situation:
- unstylish modification of a file
- stage said file for commit
- try to commit and fail due to style
- fix style, forgetting staging changes
- try to commit and fail, as although the changes staged are not
styly, the current content of the file is.

Change-Id: I5cc3f783375d9e4162e310e176103ebbf0a59023
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
[andreas.sandberg@arm.com: Rebased ontop of latest gem5]

style: Make the style fixers safe

Adds a wrapper to the fix functions of the verifiers. This wrapper first
copies the original file to a backup file, then performs the fix. If an
error occurs, the backup file is used to restore the original file.

Also fixed a line-length error in verifiers.py

style: Fix incorrect references style verifiers

The style checker for spacing around control statements (ControlSpace)
and the whitespace checker (Whitespace) didn't refer to some of their
configuration variables correctly. This changeset fixes those issues.

Reported-by: Jason Lowe-Power <power.jg@gmail.com>

style: catch trailing white spaces in make and dts files

Change-Id: I2a4f1893919660e51599902b972a6f3f5717e305
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

style: remove extra newline from white space verifier fix method

Change-Id: I7bce7d1cb04efe20d31445eb67ea5ffd2a4a41f4
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

style: Move the last bits of file_types.py to the style package

The commit that refactored the style checkers into a new Python
package (style: Refactor the style checker as a Python package)
accidentally left a fragment of file_types.py in the old location
(util/style.py). This was caused by a race between the commit that
moved the file and Nate's commit that added a copyright header to the
file.

This commit moves the last fragment (the copyright header) and removes
the old file.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

style: ignore test data in style checks

Change-Id: If797eaf3842b5c1604942bb60f091800ee814a2a
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

style: improve compatibility with old git versions

Older versions of git need the '=' to connect --diff-filter to
its argument.

Change-Id: Ic62057567db061684be88a7c2d80a6a5d4c11dcf

style: Fix Python 2.6 compatibility

The style checker code needs to disable autojunk when diffing source
files using Python's difflib. Support for this was only introduced in
Python 2.7, which leads to a TypeError exception on older Python
version. This changeset adds a fallback mechanism for old Python
versions.

style: Change maximum line length to 79 characters

The old style guide used to mandate 78 characters as the maximum line
length to accommodate traditional diffs on 80-column terminals. This
is an uncommon use case and it has therefore been decided (see email
thread on gem5-dev [1]) that a maximum length of 79-characters makes
more sense.

[1] http://comments.gmane.org/gmane.comp.emulators.m5.devel/29789

Signed-off-by: Andreas Sandberg <aandreas.sandberg@arm.com>
Reviewed-by: Brandon Potter <brandon.potter@amd.com>

style: Add a style checker that doesn't depend on Mercurial

The current style checker script, hgstyle.py, assumes that it is being
run from Mercurial. This means that it depends on the Mercurial Python
libraries, which aren't necessarily present if using git. This
changeset adds a new style checker script, style.py, that has
been designed to be run from the command line.

The script has support for detecting which revision control system is
used and is able to query both git and Mercurial for changes. This
enables the script to operate on modified regions and/or all of the
modified files in the repository.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>

scons: Automatically install the git style hook

Add a check in the main SConscript that installs the git pre-commit
hook in util/ if git is used.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
[andreas.sandberg@arm.com: Cleanups suggested by Steve]
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>

style: Add a git pre-commit hook

Add a git pre-commit hook that verifies that files that are about to
be committed. Since git stages changes into an index and the index
contains the changes that will be committed, the style checker only
looks at the state of files in the index.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>

style: Add repository helper functions

Add an AbstractRepo class and implementations for git and Mercurial
that provide a common interface to query repository status for style
checkers. The class defines the interfaces to list modified files that
are about to be committed and methods to identify changed regions.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>

style: Remove style validators

Style validators provide a subset of the style verifier functionality
and are only exposed through the "hg m5format" command. This
functionality seems to be both redundant and unused.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

style: Add a control character checker

Add a style checker that verifies that source code doesn't contain
non-printable (control) characters. The only allowed control
characters are:

* 0x0a / \n: New line
* 0x09 / \t: Tab (the whitespace checker enforces no-tabs for C/C++ files)

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Brandon Potter <brandon.potter@amd.com>

style: Refactor the style checker as a Python package

Refactor the style checker into a Python module that can be reused by
command line tools that integrate with git. In particular:

* Create a style package in util
* Move style validators from style.py to the style/validators.py.
* Move style verifiers from style.py to the style/verifiers.py.
* Move utility functions (sort_includes, region handling,
file_types) into the style package
* Move generic code from style.py to style/style.py.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>