arch-arm: Implement ARMv8.1-PAN, Privileged access never

ARMv8.1-PAN adds a new bit to PSTATE. When the value of this PAN state
bit is 1, any privileged data access from EL1 or EL2 to a virtual memory
address that is accessible at EL0 generates a Permission fault.
This feature is mandatory in ARMv8.1 implementations.
This feature is supported in AArch64 and AArch32 states.
The ID_AA64MMFR1_EL1.PAN, ID_MMFR3_EL1.PAN, and ID_MMFR3.PAN fields
identify the support for ARMv8.1-PAN.

Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Change-Id: I94a76311711739dd2394c72944d88ba9321fd159
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19729
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

arch-arm: Use ExceptionLevel type in TlbEntry

Replacing uint8_t with ExceptionLevel type in the arm TlbEntry. The
variable is representing the translation regime it is targeting.

Change-Id: Ifcd6e86c5d73f752e8476a2b7fda9ea74a0c7a3b
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19488
Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

arch-arm: Remove un-needed hyp flag in TLBI operations

The hyp flag was probably a legacy pre-v8 flag distinguishing
invalidation targeting PL2 translation regime (hyp mode).
Since the introduction of target_el parameter, hyp boolean is not needed
anymore. The patch works by setting the hyp flag in the flush* methods
in the TLB automatically by checking if target_el == EL2.

Change-Id: I798009e09ff24a383dea871e348188bae2685e8e
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Jan-Peter Larsson <jan-peter.larsson@arm.com>
Reviewed-by: Anouk Van Laer <anouk.vanlaer@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/18389
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

arch, cpu, dev, gpu, mem, sim, python: start using getPort.

Replace the getMasterPort, getSlavePort, and getEthPort functions
with getPort, and remove extraneous mechanisms that are no longer
necessary.

Change-Id: Iab7e3c02d2f3a0cf33e7e824e18c28646b5bc318
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17040
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>

mem-cache: alias to mem::getMasterPort in TLB class

TLB:getMasterPort is used to obtain the PageWalkMasterPort if present and
hides the BaseTLB::getMasterPort().

The TLB::getMasterPort() is renamed according to the expected behavior.

Change-Id: If4f61189094a706d59805cd10f4f814e5830eda8
Reviewed-on: https://gem5-review.googlesource.com/c/16648
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>

arch, arm: Effect of AT instructions on descriptor handling

Some address translation instructions will stop translation after
the 1st stage and intercept the IPA, even in the presence of
stage 2 (eg AT S1E1). However, in the case of a TLB miss, the
table descriptors still need to be translated from IPA to PA to
avoid fetching the wrong addresses. This commit splits whether
IPA->PA translation is required for the VA and/or for the table
descriptors.

Change-Id: Ie53cdc00585f116150256f1d833460931b3bfb7d
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/13781
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>

misc: Using smart pointers for memory Requests

This patch is changing the underlying type for RequestPtr from Request*
to shared_ptr<Request>. Having memory requests being managed by smart
pointers will simplify the code; it will also prevent memory leakage and
dangling pointers.

Change-Id: I7749af38a11ac8eb4d53d8df1252951e0890fde3
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/10996
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

arch-arm: Respect EL from translation type

There are cases where instructions request translations in the context
of a lower EL. This is currently not respected in the TLB and the page
table walker. Fix that.

Change-Id: Icd59657a1ecfd8bd75a001bb1a4e41a6f4808a36
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/10506
Maintainer: Giacomo Travaglini <giacomo.travaglini@arm.com>

arch-arm: Distinguish IS TLBI from non-IS

TLBI broadcasting was the default implementation of most of TLBI
instructions. This patch applies the broadcasting behaviour only to the
Inner-Shareable subset, while simpler TLB invalidation instructions only
affect the PE that executes them.

Change-Id: Idb01d0d4f593131f657e8fc9668112de8e4ccdcb
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/9182
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>

arm: Make translateFunctional override the base implementation.

Now that translateFunctional is a virtual function, having an extra
parameter with a default value makes the compiler fall through to the
base implementation instead of overriding it. This change removes the
default value for the extra parameter, and adds a small wrapper with
the correct signature which overrides the base implementation and calls
the full version with the previously default value for the extra
parameter. To callers this will look like the same thing, but the
the right function will get called.

This was what was already being done for transateAtomic and
translateTiming.

Change-Id: I0b71adf34fd6f326005edbb8eaac93275b437c55
Reviewed-on: https://gem5-review.googlesource.com/7121
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>

arch,cpu: "virtualize" the TLB interface.

CPUs have historically instantiated the architecture specific version
of the TLBs to avoid a virtual function call, making them a little bit
more dependent on what the current ISA is. Some simple performance
measurement, the x86 twolf regression on the atomic CPU, shows that
there isn't actually any performance benefit, and if anything the
simulator goes slightly faster (although still within margin of error)
when the TLB functions are virtual.

This change switches everything outside of the architectures themselves
to use the generic BaseTLB type, and then inside the ISA for them to
cast that to their architecture specific type to call into architecture
specific interfaces.

The ARM TLB needed the most adjustment since it was using non-standard
translation function signatures. Specifically, they all took an extra
"type" parameter which defaulted to normal, and translateTiming
returned a Fault. translateTiming actually doesn't need to return a
Fault because everywhere that consumed it just stored it into a
structure which it then deleted(?), and the fault is stored in the
Translation object when the translation is done.

A little more work is needed to fully obviate the arch/tlb.hh header,
so the TheISA::TLB type is still visible outside of the ISAs.
Specifically, the TlbEntry type is used in the generic PageTable which
lives in src/mem.

Change-Id: I51b68ee74411f9af778317eff222f9349d2ed575
Reviewed-on: https://gem5-review.googlesource.com/6921
Maintainer: Gabe Black <gabeblack@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

arm: Add support for memory-mapped m5ops

Add support for a memory mapped m5op interface. When enabled, the TLB
intercepts accesses in the 64KiB region designated by the
ArmTLB.m5ops_base parameter. An access to this range maps to a
specific m5op call. The upper 8 bits of the offset into the range
denote the m5op function to call and the lower 8 bits denote the
subfunction.

Change-Id: I55fd8ac1afef4c3cc423b973870c9fe600a843a2
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2964

arm: Add TLBI instruction for stage 2 IPA's

This patch adds support for stage 2 TLBI instructions
such as TLBI IPAS2E1_Xt.

Change-Id: I0cd5e8055b0c1003e03439aa5183252f50ea0a88

arm: Fix EL perceived at TLB for address translation instructions

During address translation instructions (such as AT S1E1R_Xt) the exception
level can be different than the current exception level. This patch fixes
how the TLB determines what EL to use during these instructions.

Change-Id: Ia9ce229404de9e284bc1f7479fd2c580efd55f8f

arm: Refactor the TLB test interface

Refactor the TLB and page table walker test interface to use a dynamic
registration mechanism. Instead of patching a couple of empty methods
to wire up a TLB tester, this change allows such testers to register
themselves using the setTestInterface() method.

misc: Add explicit overrides and fix other clang >= 3.5 issues

This patch adds explicit overrides as this is now required when using
"-Wall" with clang >= 3.5, the latter now part of the most recent
XCode. The patch consequently removes "virtual" for those methods
where "override" is added. The latter should be enough of an
indication.

As part of this patch, a few minor issues that clang >= 3.5 complains
about are also resolved (unused methods and variables).

misc: Remove redundant compiler-specific defines

This patch moves away from using M5_ATTR_OVERRIDE and the m5::hashmap
(and similar) abstractions, as these are no longer needed with gcc 4.7
and clang 3.1 as minimum compiler versions.

arm: Change TLB Software Caching

In ARM, certain variables are only updated when a necessary change is
detected. Having 2 SMT threads share a TLB resulted in these not being
updated as required. This patch adds a thread context identifer to
assist in the invalidation of these variables.

sim: Refactor and simplify the drain API

The drain() call currently passes around a DrainManager pointer, which
is now completely pointless since there is only ever one global
DrainManager in the system. It also contains vestiges from the time
when SimObjects had to keep track of their child objects that needed
draining.

This changeset moves all of the DrainState handling to the Drainable
base class and changes the drain() and drainResume() calls to reflect
this. Particularly, the drain() call has been updated to take no
parameters (the DrainManager argument isn't needed) and return a
DrainState instead of an unsigned integer (there is no point returning
anything other than 0 or 1 any more). Drainable objects should return
either DrainState::Draining (equivalent to returning 1 in the old
system) if they need more time to drain or DrainState::Drained
(equivalent to returning 0 in the old system) if they are already in a
consistent state. Returning DrainState::Running is considered an
error.

Drain done signalling is now done through the signalDrainDone() method
in the Drainable class instead of using the DrainManager directly. The
new call checks if the state of the object is DrainState::Draining
before notifying the drain manager. This means that it is safe to call
signalDrainDone() without first checking if the simulator has
requested draining. The intention here is to reduce the code needed to
implement draining in simple objects.

sim: Refactor the serialization base class

Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:

* Add a set of APIs to serialize into a subsection of the current
object. Previously, objects that needed this functionality would
use ad-hoc solutions using nameOut() and section name
generation. In the new world, an object that implements the
interface has the methods serializeSection() and
unserializeSection() that serialize into a named /subsection/ of
the current object. Calling serialize() serializes an object into
the current section.

* Move the name() method from Serializable to SimObject as it is no
longer needed for serialization. The fully qualified section name
is generated by the main serialization code on the fly as objects
serialize sub-objects.

* Add a scoped ScopedCheckpointSection helper class. Some objects
need to serialize data structures, that are not deriving from
Serializable, into subsections. Previously, this was done using
nameOut() and manual section name generation. To simplify this,
this changeset introduces a ScopedCheckpointSection() helper
class. When this class is instantiated, it adds a new /subsection/
and subsequent serialization calls during the lifetime of this
helper class happen inside this section (or a subsection in case
of nested sections).

* The serialize() call is now const which prevents accidental state
manipulation during serialization. Objects that rely on modifying
state can use the serializeOld() call instead. The default
implementation simply calls serialize(). Note: The old-style calls
need to be explicitly called using the
serializeOld()/serializeSectionOld() style APIs. These are used by
default when serializing SimObjects.

* Both the input and output checkpoints now use their own named
types. This hides underlying checkpoint implementation from
objects that need checkpointing and makes it easier to change the
underlying checkpoint storage code.

arm: Cleanup arch headers to remove dma_device.hh dependency

Break the dependency on dma_device.hh by forward-declaring DmaPort in
the relevant header.

arm: Make address translation faster with better caching

This patch adds better caching of the sys regs for AArch64, thus
avoiding unnecessary calls to tc->readMiscReg(MISCREG_CPSR) in the
non-faulting case.

arm: Remove unnecessary boot uncachability

With the recent patches addressing how we deal with uncacheable
accesses there is no longer need for the work arounds put in place to
enforce certain sections of memory to be uncacheable during boot.

arm: Share a port for the two table walker objects

This patch changes how the MMU and table walkers are created such that
a single port is used to connect the MMU and the TLBs to the memory
system. Previously two ports were needed as there are two table walker
objects (stage one and stage two), and they both had a port. Now the
port itself is moved to the Stage2MMU, and each TableWalker is simply
using the port from the parent.

By using the same port we also remove the need for having an
additional crossbar joining the two ports before the walker cache or
the L2. This simplifies the creation of the CPU cache topology in
BaseCPU.py considerably. Moreover, for naming and symmetry reasons,
the TLB walker port is connected through the stage-one table walker
thus making the naming identical to x86. Along the same line, we use
the stage-one table walker to generate the master id that is used by
all TLB-related requests.

sim: Move the BaseTLB to src/arch/generic/

The TLB-related code is generally architecture dependent and should
live in the arch directory to signify that.

arch: Use shared_ptr for all Faults

This patch takes quite a large step in transitioning from the ad-hoc
RefCountingPtr to the c++11 shared_ptr by adopting its use for all
Faults. There are no changes in behaviour, and the code modifications
are mostly just replacing "new" with "make_shared".

arm: Add TLB PMU probes

This changeset adds probe points that can be used to implement PMU
counters for TLB stats. The following probes are supported:

* ArmISA::TLB::ppRefills / TLB Refills (TLB insertions)

arch, arm: Preserve TLB bootUncacheability when switching CPUs

The ARM TLBs have a bootUncacheability flag used to make some loads
and stores become uncacheable when booting in FS mode. Later the
flag is cleared to let those loads and stores operate as normal. When
doing a takeOverFrom(), this flag's state is not preserved and is
momentarily reset until the CPSR is touched. On single core runs this
is a non-issue. On multi-core runs this can lead to crashes on the O3
CPU model from the following series of events:
1) takeOverFrom executed to switch from Atomic -> O3
2) All bootUncacheability flags are reset to true
3) Core2 tries to execute a load covered by bootUncacheability, it
is flagged as uncacheable
4) Core2's load needs to replay due to a pipeline flush
3) Core1 core does an action on CPSR
4) The handling code for CPSR then checks all other cores
to determine if bootUncacheability can be set to false
5) Asynchronously set bootUncacheability on all cores to false
6) Core2 replays load previously set as uncacheable and notices
it is now flagged as cacheable, leads to a panic.
This patch implements takeOverFrom() functionality for the ARM TLBs
to preserve flag values when switching from atomic -> detailed.

arm: Add support for ARMv8 (AArch64 & AArch32)

Note: AArch64 and AArch32 interworking is not supported. If you use an AArch64
kernel you are restricted to AArch64 user-mode binaries. This will be addressed
in a later patch.

Note: Virtualization is only supported in AArch32 mode. This will also be fixed
in a later patch.

Contributors:
Giacomo Gabrielli (TrustZone, LPAE, system-level AArch64, AArch64 NEON, validation)
Thomas Grocutt (AArch32 Virtualization, AArch64 FP, validation)
Mbou Eyole (AArch64 NEON, validation)
Ali Saidi (AArch64 Linux support, code integration, validation)
Edmund Grimley-Evans (AArch64 FP)
William Wang (AArch64 Linux support)
Rene De Jong (AArch64 Linux support, performance opt.)
Matt Horsnell (AArch64 MP, validation)
Matt Evans (device models, code integration, validation)
Chris Adeniyi-Jones (AArch64 syscall-emulation)
Prakash Ramrakhyani (validation)
Dam Sunwoo (validation)
Chander Sudanthi (validation)
Stephan Diestelhorst (validation)
Andreas Hansson (code integration, performance opt.)
Eric Van Hensbergen (performance opt.)
Gabe Black

arch: Create a method to finalize physical addresses
in the TLB

Some architectures (currently only x86) require some fixing-up of
physical addresses after a normal address translation. This is usually
to remap devices such as the APIC, but could be used for other memory
mapped devices as well. When running the CPU in a using hardware
virtualization, we still need to do these address fix-ups before
inserting the request into the memory system. This patch moves this
patch allows that code to be used by such CPUs without doing full
address translations.

arm: fix a page table walker issue where a page could be translated multiple times

If multiple memory operations to the same page are miss the TLB they are
all inserted into the page table queue and before this change could result
in multiple uncessesary walks as well as duplicate enteries being inserted
into the TLB.

arm: Invalidate cached TLB configuration in drainResume

Currently, we invalidate the cached miscregs in
TLB::unserialize(). The intended use of the drainResume() method is to
invalidate cached state and prepare the system to resume after a CPU
handover or (un)serialization. This patch moves the TLB miscregs
invalidation code to the drainResume() method to avoid surprising
behavior.

Port: Add protocol-agnostic ports in the port hierarchy

This patch adds an additional level of ports in the inheritance
hierarchy, separating out the protocol-specific and protocl-agnostic
parts. All the functionality related to the binding of ports is now
confined to use BaseMaster/BaseSlavePorts, and all the
protocol-specific parts stay in the Master/SlavePort. In the future it
will be possible to add other protocol-specific implementations.

The functions used in the binding of ports, i.e. getMaster/SlavePort
now use the base classes, and the index parameter is updated to use
the PortID typedef with the symbolic InvalidPortID as the default.

MEM: Introduce the master/slave port sub-classes in C++

This patch introduces the notion of a master and slave port in the C++
code, thus bringing the previous classification from the Python
classes into the corresponding simulation objects and memory objects.

The patch enables us to classify behaviours into the two bins and add
assumptions and enfore compliance, also simplifying the two
interfaces. As a starting point, isSnooping is confined to a master
port, and getAddrRanges to slave ports. More of these specilisations
are to come in later patches.

The getPort function is not getMasterPort and getSlavePort, and
returns a port reference rather than a pointer as NULL would never be
a valid return value. The default implementation of these two
functions is placed in MemObject, and calls fatal.

The one drawback with this specific patch is that it requires some
code duplication, e.g. QueuedPort becomes QueuedMasterPort and
QueuedSlavePort, and BusPort becomes BusMasterPort and BusSlavePort
(avoiding multiple inheritance). With the later introduction of the
port interfaces, moving the functionality outside the port itself, a
lot of the duplicated code will disappear again.

Merge with head, hopefully the last time for this batch.

ARM: Turn on the page table walker on ARM in SE mode.

CheckerCPU: Re-factor CheckerCPU to be compatible with current gem5

Brings the CheckerCPU back to life to allow FS and SE checking of the
O3CPU. These changes have only been tested with the ARM ISA. Other
ISAs potentially require modification.

ARM: update TLB to set request packet ASID field

ARM: Mark some variables uncacheable until boot all CPUs are enabled.

There are a set of locations is the linux kernel that are managed via
cache maintence instructions until all processors enable their MMUs & TLBs.
Writes to these locations are manually flushed from the cache to main
memory when the occur so that cores operating without their MMU enabled
and only issuing uncached accesses can receive the correct data. Unfortuantely,
gem5 doesn't support any kind of software directed maintence of the cache.
Until such time as that support exists this patch marks the specific cache blocks
that need to be coherent as non-cacheable until all CPUs enable their MMU and
thus allows gem5 to boot MP systems with caches enabled (a requirement for
booting an O3 cpu and thus an O3 CPU regression).

ARM: Better RealView/Versatile EB platform support.

Add registers and components to better support the VersatileEB board.
Made the MIDR and SYS_ID register parameters to ArmSystem and RealviewCtrl
respectively.

includes: sort all includes

Fault: Rename sim/fault.hh to fault_fwd.hh to distinguish it from faults.hh.

Make commenting on close namespace brackets consistent.

Ran all the source files through 'perl -pi' with this script:

s|\s*(};?\s*)?/\*\s*(end\s*)?namespace\s*(\S+)\s*\*/(\s*})?|} // namespace $3|;
s|\s*};?\s*//\s*(end\s*)?namespace\s*(\S+)\s*|} // namespace $2\n|;
s|\s*};?\s*//\s*(\S+)\s*namespace\s*|} // namespace $1\n|;

Also did a little manual editing on some of the arch/*/isa_traits.hh files
and src/SConscript.

Style: Replace some tabs with spaces.

ARM: Support switchover with hardware table walkers

ARM: Cache the misc regs at the TLB to limit readMiscReg() calls.

ARM: Add some TLB statistics for ARM

ARM: Add checkpointing support

ARM: Make the TLB a little bit faster by moving most recently used items to front of list

ARM: Implement functional virtual to physical address translation
for debugging and program introspection.

Faults: Pass the StaticInst involved, if any, to a Fault's invoke method.

Also move the "Fault" reference counted pointer type into a separate file,
sim/fault.hh. It would be better to name this less similarly to sim/faults.hh
to reduce confusion, but fault.hh matches the name of the type. We could change
Fault to FaultPtr to match other pointer types, and then changing the name of
the file would make more sense.

ARM: Implement all ARM SIMD instructions.

MEM: Make CLREX a first class request operation and clear locks in caches when it in received

ARM: Implement CLREX

stats: only consider a formula initialized if there is a formula

ARM: Added support for Access Flag and some CP15 regs (V2PCWPR, V2PCWPW, V2PCWUR, V2PCWUW,...)

ARM: Some TLB bug fixes.

ARM: Implement the ARM TLB/Tablewalker. Needs performance improvements.

ARM: Start over with translation from Alpha code as opposed to something that has cruft from 4 different ISAs.

ARM: Implement the V7 version of alignment checking.

arm: Unify the ARM tlb. We forgot about this when we did the rest.
This code compiles, but there are no tests still

Merge ARM into the head. ARM will compile but may not actually work.

arm: add ARM support to M5