mem-cache: mark block as dirty when handling SW prefetch

This addresses the issue described in
64687ee mem-cache: Mark block as dirty after a SWPrefetchEXResp.

Previous patch misses cases when the prefetch response is ReadExResp or
UpgradeResp. Also, marking the block as dirty in serviceMSHRTargets
instead of in handleFill covers cases when the prefetch is coalesced with
other requests.

Change-Id: I2b377fdd240eb0f09e720b6bb284dee6545925ce
Signed-off-by: Tiago Mück <tiago.muck@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19688
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

Revert "mem-cache: Remove writebacks packet list"

This reverts commit bf0a722acdd8247602e83720a5f81a0b69c76250.

Reason for revert: This patch introduces a bug:

The problem here is that the insertion of block A may cause the
eviction of block B, which on the lower level may cause the
eviction of block A. Since A is not marked as present yet, A is
"safely" removed from the snoop filter

However, by reverting it, using atomic and a Tags sub-class that
can generate multiple evictions at once becomes broken when using
Atomic mode and shall be fixed in a future patch.

Change-Id: I5b27e54b54ae5b50255588835c1a2ebf3015f002
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19088
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

cpu,mem: Add support for partial loads/stores and wide mem. accesses

This changeset adds support for partial (or masked) loads/stores, i.e.
loads/stores that can disable accesses to individual bytes within the
target address range. In addition, this changeset extends the code to
crack memory accesses across most CPU models (TimingSimpleCPU still
TBD), so that arbitrarily wide memory accesses are supported. These
changes are required for supporting ISAs with wide vectors.

Additional authors:
- Gabor Dozsa <gabor.dozsa@arm.com>
- Tiago Muck <tiago.muck@arm.com>

Change-Id: Ibad33541c258ad72925c0b1d5abc3e5e8bf92d92
Signed-off-by: Giacomo Gabrielli <giacomo.gabrielli@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/13518
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Remove writebacks packet list

Previously all atomic writebacks concerned a single block,
therefore, when a block was evicted, no other block would be
pending eviction. With sector tags (and compression),
however, a single replacement can generate many evictions.

This can cause problems, since a writeback that evicts a block
may evict blocks in the lower cache. If one of these conflict
with one of the blocks pending eviction in the higher level, the
snoop must inform it to the lower level. Since atomic mode does
not have a writebuffer, this kind of conflict wouldn't be noticed.

Therefore, instead of evicting multiple blocks at once, we
do it one by one.

Change-Id: I2fc2f9eb0f26248ddf91adbe987d158f5a2e592b
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/18209
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Add compression and decompression calls

Add a compressor to the base cache class and compress within
block allocation and decompress on writebacks.

This change does not implement data expansion (fat writes) yet,
nor it adds the compression latency to the block write time.

Change-Id: Ie36db65f7487c9b05ec4aedebc2c7651b4cb4821
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/11410
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Add packet matching functions

Add both block and non-block-aligned packet matching functions,
so that both address and secure bits are checked when checking
whether a packet matches a request.

Change-Id: Id0069befb925d112e06f250741cb47d9dfa249cc
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17533
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Move Target to QueueEntry

WriteQueueEntry's target has 100% functionality overlap with MSHR's,
therefore make it base to MSHR::Target.

Change-Id: I48614e78179d708bd91bbe75a752e5a05146e8eb
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17534
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Fix MSHR handling of cache clean requests

Previously satisfied clean requests would not snoop in-service
MSHRs. This is a problem when a clean request is also invalidating, in
which case we have to post-invalidate or post-downgrade outstanding
requests. This changes fixes this bug.

Change-Id: I31e42aa94dd3637b2818e00fbaae68c810145eaf
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17728
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem-cache: Copy over flags to forwarded response

A cache that forwards a request to the memory below does not fill and
forwards the response with the data to cache above. This change
ensures that the flags of the original response are also preserved.

Change-Id: I244b20b073c31b976358816c5b14bba413b8271f
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/16182
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem: Determine if a packet queue forces ordering at construction

A packet queue is typically used to hold on to packets that are
schedules to be sent in the future or when they need to queue behind
younger packets that have been sent out yet. Due to memory order
requirements, some MemObjects need to maintain the order for packet
(mostly responses) that reference the same cache block.

Prior to this patch the ordering requirements where determined when
the packet was scheduled to be sent. This patch moves the parameter to
the constructor.

Change-Id: Ieb4d94e86bc7514f5036b313ec23ea47dd653164
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/15555
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Remove writebacks parameter from serviceMSHRTargets

Change 8ba77ae8fc98a355082da2bd9fdc6ecf4928f725 introduced the
writebacks parameter, but it was never used.

Change-Id: I225e5b399de42d77c72fc0012d3dc93ef39b8853
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/14896
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Align how we handle requests in atomic with timing

Requests, for which a cache has already committed to respond do not
perform any lookups. Previously in atomic mode the packet would pay
the lookup latency while in timing it wouldn't. This patch aligns
recvAtomic with recvTimingReq and removes the lookup latency from the
the handling of such requests.

Change-Id: I50a0631f8058e5086d94d55af0e1788a60e2883f
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/c/14175
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Move evictBlock(CacheBlk*, PacketList&) to base

Move evictBlock(CacheBlk*, PacketList&) to base cache,
as it is both sub-classes implementations are equal.

Change-Id: I80fbd16813bfcc4938fb01ed76abe29b3f8b3018
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/13656
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Add write coalescing and write-no-allocate to the caches

Enable the cache to detect contiguous writes and hold on to the MSHR
long enough to allow the entire line to be written. If the whole line
is written, the MSHR will be sent out as an invalidation requests, as
it is part of a whole-line write, i.e. no-fetch-on-write.

The cache is also able to switch to a write-no-allocate policy on the
actual completion of the writes, and instead use the tempBlock and
turn the write operation into a writeback.

These policies are all well-known, and described in works such as
Jouppi, Cache Write Policies and Performance, vol 21, no 2, ACM, 1993.

Change-Id: I19792f2970b3c6798c9b2b493acdd156897284ae
Reviewed-on: https://gem5-review.googlesource.com/c/12907
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Restructure whole-line writes to simplify write merging

This patch changes how we deal with whole-line writes their
responses. With these changes, we use the MSHR tracking to determine
if a whole-line is written, and on a fill we simply handle the
invalidation response, with the actual writes taking place as part of
satisfying the CPU-side hit.

Change-Id: I9a18e41a95db3c20b97f8bca7d95ff33d35a578b
Reviewed-on: https://gem5-review.googlesource.com/c/12905
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Rename blk.cc/hh to cache_blk.cc/hh

Rename the files blk.cc and blk.hh to cache_blk.cc and cache_blk.hh
to comply with the usual file-class naming rules.

Change-Id: I8af45df3e4b8dd934fd9929ec914fb230cb2cb09
Signed-off-by: Daniel R. Carvalho <odanrc@yahoo.com.br>
Reviewed-on: https://gem5-review.googlesource.com/c/13416
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Fix bug in handleAtomicReqMiss

"4976ff5 mem-cache: Refactor the recvAtomic function" introduced a bug
where if an atomic request that fills in using the tempBlock it will
not evict it when it finishes handling the request as it should. This
triggers an assertion. This change fixes this bug.

Change-Id: I73c808a7e15237eddb36b5448ef6728f7bcf7fd9
Reviewed-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/12644
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Promote deferred targets on cache clean responses

While a cache clean operation is pending, all requests to the
corresponding block get deferred. When the response of a cache clean
operation is received, if the block is present and the response is not
invalidating, we can service all deferred targets that didn't require
writable. This change implements this functionality.

Change-Id: Ief47e74d07749a6a9736ab450eb46eefa53464a2
Reviewed-on: https://gem5-review.googlesource.com/11018
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>

misc: Using smart pointers for memory Requests

This patch is changing the underlying type for RequestPtr from Request*
to shared_ptr<Request>. Having memory requests being managed by smart
pointers will simplify the code; it will also prevent memory leakage and
dangling pointers.

Change-Id: I7749af38a11ac8eb4d53d8df1252951e0890fde3
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/10996
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

misc: Substitute pointer to Request with aliased RequestPtr

Every usage of Request* in the code has been replaced with the
RequestPtr alias. This is a preparing patch for when RequestPtr will be
the typdefed to a smart pointer to Request rather then a raw pointer to
Request.

Change-Id: I73cbaf2d96ea9313a590cdc731a25662950cd51a
Signed-off-by: Giacomo Travaglini <giacomo.travaglini@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/10995
Reviewed-by: Anthony Gutierrez <anthony.gutierrez@amd.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Anthony Gutierrez <anthony.gutierrez@amd.com>

mem-cache: Create an address aware TempCacheBlk

tempBlock has its member variables manually set in order to allow
it to be used in the block address regeneration function. This is
not necessary, and ti can be simply given the address, so it does
not need to be aware of set and tag. This will simplify
implementation of sector and skewed caches.

Change-Id: Iaffb10c323509722cd5589fe1030b818d43336d6
Reviewed-on: https://gem5-review.googlesource.com/9961
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Fix secure bit modification

Secure bit was being updated outside insertion.

Change-Id: I83d9b010e8cf64013bbea9bae3ea68b0c414a189
Reviewed-on: https://gem5-review.googlesource.com/10622
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Move cache bypass mechanism to the ports

Cache bypass is necessary for cpu models like the KvmCPU. Previously
the bypass would happen at the cache classes. With this change the
bypassing happens directly at the ports.

Change-Id: I34de9fc63383aee8590643e169501ea6060d2d62
Reviewed-on: https://gem5-review.googlesource.com/10432
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Adopt a more sensible cache class hierarchy

This patch changes what goes into the BaseCache and what goes into the
Cache, to make it easier to add a NoncoherentCache with as much re-use
as possible. A number of redundant members and definitions are also
removed in the process.

This is a modified version of a changeset put together by Andreas
Hansson <andreas.hansson@arm.com>

Change-Id: Ie9dd73c4ec07732e778e7416b712dad8b4bd5d4b
Reviewed-on: https://gem5-review.googlesource.com/10431
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Add helper function to perform evictions

Change-Id: I2df24eb1a8516220bec9b685c8c09bf55be18681
Reviewed-on: https://gem5-review.googlesource.com/10430
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Delegate block invalidation to block allocation

For a block replacement we first select a victim block, we invalidate
it and then populate it with the new information. Prior to this change
BaseTags::insertBlock() did the invalidation and filled in the block
with the new information. Now that the replacements stat is moved to
the BaseCache, insertBlock does not need to perform the invalidation
and as a result we can unify the block eviction code in BaseCache.

Change-Id: I5bdf00b2dab2752ed2137ab7201ed1dc451333b3
Reviewed-on: https://gem5-review.googlesource.com/10429
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem-cache: Refactor the recvAtomic function

The recvAtomic function in the cache handles atomic requests. Over
time, recvAtomic has grown in complexity and code size. This change
factors out some of its functionality in a separate functiona. The new
functions handles atomic requests that miss.

Change-Id: If77d2de1e3e802e1da37f889f68910e700c59209
Reviewed-on: https://gem5-review.googlesource.com/10425
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Refactor the cache recvTimingReq function

The recvTimingReq function in the cache handles timing requests. Over
time, recvTimingReq has grown in complexity and code size. This change
factors out some of its functionality in two separate functions. The
new functions handle timing requests that hit and timing requests that
miss separately.

Change-Id: I09902d648d7272f0f9ec2851fa6376f7305ba418
Reviewed-on: https://gem5-review.googlesource.com/10424
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Refactor the cache recvTimingResp function

The recvTimingResp function in the cache handles timing
responses. Over time, recvTimingResp has grown in complexity and code
size. This change factors out some of its functionality to a separate
function. The new function iterates through the in-service targets and
handles them accordingly.

Change-Id: I0ef28288640f6be1b30452b0664d32432e692ea6
Reviewed-on: https://gem5-review.googlesource.com/10423
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Determine if an MSHR has requests from another cache

To decide whether we allocate upon receiving a response we need to
determine if any of the currently serviced requests (non-deferred
targets) is comming from another cache. This change adds support for
tracking this information in the MSHR.

Change-Id: If1db93c12b6af5813b91b9d6b6e5e196d327f038
Reviewed-on: https://gem5-review.googlesource.com/10422
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Move replacements stat to the base cache class

Change-Id: I25dbcfcddfe1c422a76eb1af3f726c1360d8d110
Reviewed-on: https://gem5-review.googlesource.com/10426
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Simplify writeback for the tempBlock in recvTimingResp

When we use the tempBlock to fill-in, we have to write it back and
invalidate it at the end of current transaction. This patch simplifies
the writeback flow by treating it as a regular writeback.

Change-Id: I257be7bbff211e2832ad001a4e991daf67704485
Reviewed-on: https://gem5-review.googlesource.com/10421
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Create block insertion function

Create a block insertion function to be used when inserting
blocks. This resets the number of references to 1 (the
insertion is taken into account), sets the insertion tick,
and set secure state.

Change-Id: Ifc34cbbd1c125207ce47912d188809221c7a157e
Reviewed-on: https://gem5-review.googlesource.com/9824
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Use Packet functions to write data blocks

Instead of using raw memcpy, use the proper writer functions
from the Packet class in Cache.

Fixed typos in comments of these functions.

Change-Id: I156a00989c6cbaa73763349006a37a18243d6ed4
Reviewed-on: https://gem5-review.googlesource.com/9661
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Remove unused return value from the recvTimingReq func

The recvTimingReq function in the cache always returns true. This
changeset removes the return value.

Change-Id: I00dddca65ee7224ecfa579ea5195c841dac02972
Reviewed-on: https://gem5-review.googlesource.com/8289
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem-cache: Split array indexing and replacement policies.

Replacement policies (LRU, Random) are currently considered as array
indexing methods, but have completely different functionalities:

- Array indexers determine the possible locations for block allocation.
This information is used to generate replacement candidates when
conflicts happen.
- Replacement policies determine which of the replacement candidates
should be evicted to make room for new allocations.

For this reason, they were split into different classes. Advantages:

- Easier and more straightforward to implement other replacement
policies (RRIP, LFU, ARC, ...)
- Allow easier future implementation of cache organization schemes

As now we can't assure the use of sets, the previous way to create a
true LRU is not viable. Now a timestamp_bits parameter controls how
many bits are dedicated for the timestamp, and a true LRU can be
achieved through an infinite number of bits (although a few bits suffice
in practice).

Change-Id: I23750db121f1474d17831137e6ff618beb2b3eda
Reviewed-on: https://gem5-review.googlesource.com/8501
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Allow clean operations when block allocation fails

Block allocation can fail when there is an in-service MSHR that
operates on the victim block. This can happed due to:
* an upgrade operation: a request that needs a writable copy of the
block finds a shared (non-writable) copy of the block in the cache
and has allocates an MSHR for the pending upgrade operation, or
* a clean operation: a clean request finds a dirty copy of the block
and allocates an MSHR for the pending clean operation.
This changes relaxes an assertion to allow for the 2nd case (cache
clean operations).

Change-Id: Ib51482160b5f2b3702ed744b0eac2029d34bc9d4
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/9021
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>

mem-cache: Use CacheBlk parameter on address regeneration

Skewed caches need to know the way to regenerate a block address.

Change-Id: I62c61ac9509eff2f37bad36862751956db7a6e40
Reviewed-on: https://gem5-review.googlesource.com/8782
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Populate whenReady for blocks filled from writebacks

Writebacks write data to either an existing block or a newly allocated
block. In either case we need to populate the whenReady field of the
block which will determine when the new value can be used.

Change-Id: I5788fad0b8086a1be96714639bf6a9470b334926
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/8285
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Populate the secure bit when the temp block is filled

The secure bit should be set when we fill a block with data from a
secure location, as indicated by the packet that triggers the fill.
This patch fixes a bug in which the cache wouldn't populate the secure
bit when filling the temp block.

Change-Id: I95c706146449804ff42b205b25dd79750f3e882a
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/8284
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem-cache: Remove unnecessary block initialization on writeback

Change-Id: Ia9b825bcbb8d326705f74c15a93a88703153ba5a
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/8283
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>

mem-cache: Cleaned blocks should be marked as not writable

A writeclean packet writes a dirty block to the memory below and
therefore sets the dirty flag for the block when the memory below is a
cache. If the block was also marked as writable it can satisfy future
write requests without further requests/snoops. This can lead to
multiple copies of the same block marked as dirty which is not
allowed. This changeset clears the writable flag from the cleaned
block to prevent the cache from satisfying future write requests
without sending a downstream request.

Change-Id: I14d3c62fd33f81b1a8ba62374c8565ccab00a6fe
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/7821
Maintainer: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem-cache: Prune unnecessary writebacks in exclusive caches

Exclusive caches use the tempBlock to fill for responses from a
downstream cache. The reason for this is that they only pass the block
to the cache above without keeping a copy. When all requests are
serviced the block is immediately invalidated unless it is dirty, in
which case it has to be written back to the memory below.

To avoid unnecessary writebacks, this changeset forces mostly
exclusive caches to issuse requests that can only fetch clean data
when possible.

Reported-by: Quereshi Muhammad Avais <avais@kaist.ac.kr>

Change-Id: I01b377563f5aa3e12d22f425a04db7c023071849
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5061
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Co-ordination of CMOs in the xbar

A clean packet request serving a cache maintenance operation (CMO)
visits all memories down to the specified xbar. The visited caches
invalidate their copy (if the CMO is invalidating) and if a dirty copy
is found a write packet writes the dirty data to the memory level
below the specified xbar. A response is send back when all the caches
are clean and/or invalidated and the specified xbar has seen the write
packet.

This patch adds the following functionality in the xbar:
1) Accounts for the cache clean requests that go through the xbar
2) Generates the cache clean response when both the cache clean
request and the corresponding writeclean packet has crossed the
destination xbar.

Previously transactions in the xbar were identified using the pointer
of the original request. Cache clean transactions comprise of two
different packets, the clean request and the writeclean, and therefore
have different request pointers. This patch adds support for custom
transaction IDs that by default take the value of the request pointer
but can be overriden by the contructor. This allows the clean request
and writeclean share the same id which the coherent xbar uses to
co-ordinate them and send the response in a timely manner.

Change-Id: I80db76386a1caded38dc66e6e18f930c3bb800ff
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5051
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem: Add support for CMOs in the cache

This change adds support for maintenance operations (CMOs) in the
cache. The supported memory operations clean and/or invalidate a cache
block as specified by its VA to the specified xbar (PoU, PoC).

A cache maintenance packet visits all memories down to the specified
xbar. Caches need to invalidate their copy if it is an invalidating
CMO. If it is (additionally) a cleaning CMO and a dirty copy exists,
the cache cleans it with a WriteClean request.

Change-Id: Ibf31daa7213925898f3408738b11b1dd76c90b79
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5049
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem: Promote deferred targets only when the block is valid

When a response indicates that there are no other sharers of the
block, the cache can promote its copy of the block to writable and
potential service deferred targets even if the request didn't ask for
a writable copy.

Previously, a response would guarantee the presence of the block in
the cache. A response could either be filling, upgrading or a response
to an invalidation due to a pending whole line write. Responses to
cache maintenance invalidations break this assumption. This change
adds an extra check to make sure that the block was already valid or
that the response is filling before promoting the block.

Change-Id: I6839f683a05d4dad4205c23f365a925b7b05e366
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Anouk Van Laer <anouk.vanlaer@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5048
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem: Support for specifying the destination of a WriteClean

Previously, WriteClean packets would always write to the first memory
below unless the memory was unable to allocate in which case it would
be forwarded further below.

This change adds support for specifying the destination of a
WriteClean packet. The cache annotates the request with the specified
destination and marks the packet as write-through upon its
creation. The coherent xbar checks packets for their destination and
resets the write-through flag when necessary e.g., the coherent xbar
that is set as the PoC will reset the write-through flag for packets
to the PoC.

Change-Id: I84b653f5cb6e46e97e09508649a3725d72d94606
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Anouk Van Laer <anouk.vanlaer@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5046
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>

mem: Add support for WriteClean packets in the memory system

This change adds support for creating and handling WriteClean
packets. The WriteClean operation is almost identical to a
WritebackDirty with the exception that the cache generating a
WriteClean retains a copy of the block.

Change-Id: I63c8de62919fad0f9547d412f8266aa4292ebecd
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Anouk Van Laer <anouk.vanlaer@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/5045
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem-cache: Add support for checking whether a cache is busy

This changeset adds support for checking whether the cache is
currently busy and a timing request would be rejected.

Change-Id: I5e37b011b2387b1fa1c9e687b9be545f06ffb5f5
Reviewed-on: https://gem5-review.googlesource.com/5042
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

misc: Rename misc.(hh|cc) to logging.(hh|cc)

These files aren't a collection of miscellaneous stuff, they're the
definition of the Logger interface, and a few utility macros for
calling into that interface (panic, warn, etc.).

Change-Id: I84267ac3f45896a83c0ef027f8f19c5e9a5667d1
Reviewed-on: https://gem5-review.googlesource.com/6226
Reviewed-by: Brandon Potter <Brandon.Potter@amd.com>
Maintainer: Gabe Black <gabeblack@google.com>

mem-cache: Delete squashed HWPrefetches

Request and Packet for squashed HWPrefetches were not deleted

Change-Id: I9b66bb01b8ed6a5ddfaaa8739a68165dc4a7006c
Signed-off-by: Pau Cabre <pau.cabre@metempsy.com>
Reviewed-on: https://gem5-review.googlesource.com/4340
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Replace EventWrapper use with EventFunctionWrapper

NOTE: With this change there is a possibility for `DRAMCtrl::Rank`s
event names to not properly match the rank they were generated by. This
could occur if the public rank member is modified after the Rank's
construction. A patch would mean refactoring Rank and `DRAMCtrl`b to
privatize many of the members of Rank behind getters.

Change-Id: I7b8bd15086f4ffdfd3f40be4aeddac5e786fd78e
Signed-off-by: Sean Wilson <spwilson2@wisc.edu>
Reviewed-on: https://gem5-review.googlesource.com/3745
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Anthony Gutierrez <anthony.gutierrez@amd.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: Use pkt::getBlockAddr instead of BaseCace::blockAlign

Change-Id: I0ed4e528cb750a323facdc811dde7f0ed1ff228e
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>

mem: Remove unused arguments (asid/contex_id) from accessBlock

Change-Id: I79c2662fc81630ab321db8a75be6cd15fa07d372
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: Always use the helper function to invalidate a block

Policies like the LRU need to be notified when a block is invalidated,
the helper function does this along with invalidating the block.

Change-Id: I3ed59cf07938caa7f394ee6054b0af9e00b267ea
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: Populate the secure flag in the writeback visitor

Previously the writeback visitor would not consider and set the secure
flag for the blocks that are written back to memory. This patch fixes
this.

Change-Id: Ie1a425fa9211407a70a4343f2c6b3d073371378f
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: fix printing of 1st cache tags line

Rather than having the 1st line on the Log line and every other line on its
own, add a new line to have a common format for all of them. Makes parsing
a lot easier.

Reviewed at http://reviews.gem5.org/r/3808/

Signed-off-by: Jason Lowe-Power <jason@lowepower.com>

mem: Respond to InvalidateReq when the block is (pending) dirty

Previously when an InvalidateReq snooped a cache with a dirty block or
a pending modified MSHR, it would invalidate the block or set the
postInv flag. The cache would not send an InvalidateResp. though,
causing memory order violations. This patches changes this behavior,
making the cache with the dirty block or pending modified MSHR the
ordering point.

Change-Id: Ib4c31012f4f6693ffb137cd77258b160fbc239ca
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>

mem: Invalidate a blk when servicing the 1st invalidating target

Previously an MSHR with one or more invalidating targets would first
service all targets in the MSHR TargetList and then invalidate the
block. As a result any service snooping targets would lookup in the
cache and incorrectly find the block. This patch forces the
invalidation to happen when the first invalidating target is
encountered.

Change-Id: I9df15de24e1d351cd96f5a2c424d9a03d81c2cce
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>

mem: Allow non invalidating snoops on an InvalidateReq MSHR

This patch changes an assertion that previously assumed that a non
invalidating snoop request should never be serviced by an
InvalidateReq MSHR. The MSHR serves as the ordering point for the
snooping packet. When the InvalidateResp reaches the cache the
snooping packet snoops the caches above to find the requested
block. One or more of the caches above will have the block since
earlier it has seen a WriteLineReq.

Change-Id: I0c147c8b5d5019e18bd34adf9af0fccfe431ae07
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>

mem: Always use InvalidateReq to service WriteLineReq misses

Previously, a WriteLineReq that missed in a cache would send out an
InvalidateReq if the block lookup failed or an UpgradeReq if the
block lookup succeeded but the block had sharers. This changes ensures
that a WriteLineReq always sends an InvalidateReq to invalidate all
copies of the block and satisfy the WriteLineReq.

Change-Id: I207ff5b267663abf02bc0b08aeadde69ad81be61
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>

mem: Ensure InvalidateReq is considered isForward by MSHRs

This patch fixes an issue where an MSHR would incorrectly be perceived
to provide data to targets arriving after an InvalidateReq. To address
this the InvalidateReq is now treated as isForward, much like an
UpgradeReq that did not hit in the cache.

Change-Id: Ia878444d949539b5c33fd19f3e12b0b8a872275e
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>

mem: Make packet debug printing more uniform

Previously DPRINTFs printing information about a packet would use ad hoc
formats. This patch changes all DPRINTFs to use the print function
defined by the packet class, making the packet printing format more
uniform and easier to change.

Change-Id: Idd436a9758d4bf70c86a574d524648b2a2580970
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>

mem: Service only the 1st FromCPU MSHR target on ReadRespWithInv

A response to a ReadReq can either be a ReadResp or a
ReadRespWithInvalidate. As we add targets to an MSHR for a ReadReq we
assume that the response will be a ReadResp. When the response is
invalidating (ReadRespWithInvalidate) servicing more than one targets
can potentially violate the memory ordering. This change fixes the way
we handle a ReadRespWithInvalidate. When a cache receives a
ReadRespWithInvalidate we service only the first FromCPU target and
all the FromSnoop targets from the MSHR target list. The rest of the
FromCPU targets are deferred and serviced by a new request.

Change-Id: I75c30c268851987ee5f8644acb46f440b4eeeec2
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>

mem: Keep track of allocOnFill in the TargetList

Previously the information of whether a response was allocating or not
was a property of the MSHR. This change makes this flag a property of
the TargetList. Differernt TargetLists, e.g. the targets and the
deferred targets lists might have different values. Additionally, the
information about whether each of the target expects an allocating
response is stored inside the TargetList container. This allows for
repopulating the flag in case some of the targets are removed.

Change-Id: If3ec2516992f42a6d9da907009ffe3ab8d0d2021
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Stephan Diestelhorst <stephan.diestelhorst@arm.com>

mem: Update mostly exclusive policy even further

This patch takes yet another step in maintaining the clusivity, in
that it allows a mostly-inclusive cache to hold on to blocks even when
responding to a ReadExReq or UpgradeReq. Previously the cache simply
invalidated these blocks, but there is no strict need to do so.

The most important part of this patch is that we simply mark the block
clean when satisfying the upstream request where the cache is allowed
to keep the block. The only tricky part of the patch is in the memory
management of deferred snoops, where we need to distinguish the cases
where only the packet was copied (we expected to respond), and the
cases where we created an entirely new packet and request (we kept it
only to replay later).

The code in satisfyRequest is definitely ready for some refactoring
after this.

Change-Id: I201ddc7b2582eaa46fb8cff0c7ad09e02d64b0fc
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>

mem: Update mostly exclusive cache policy to cover more cases

This patch changes how the mostly exclusive policy is enforced to
ensure that we drop blocks when we should. As part of this change, the
actual invalidation due to the clusivity enforcement is moved outside
the hit handling, to a separate method maintainClusivity. For the
timing mode that means we can deal with all MSHR targets before taking
any action and possibly dropping the block. The method
satisfyCpuSideRequest is also renamed satisfyRequest as part of this
change (since we only ever see requests from the cpu-side port).

Change-Id: If6f3d1e0c3e7be9a67b72a55e4fc2ec4a90fd3d2
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>

mem: Add a FromCache packet attribute

This patch adds a FromCache attribute to the packet, and updates a
number of the existing request commands to reflect that the request
originates from a cache. The attribute simplifies checking if a
requests came from a cache or not, and this is used by both the cache
and snoop filter in follow-on patches.

Change-Id: Ib0a7a080bbe4d6036ddd84b46fd45bc7eb41cd8f
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Reviewed-by: Steve Reinhardt <stever@gmail.com>

mem: Remove stale argument from a DPRINTF in the cache code

Change-Id: I70dd11c23b45dfc606ef08233d2e50fcc0817505
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: Fix memory leak in handling of deferred snoops

This patch fixes a memory leak where deferred snoop packets never got
deallocated. On the call to MSHR::handleSnoop these snoops were
treated as if a response will be sent, as the MSHR was
pendingModified. Consequently, a copy of the packet was created and
added to the MSHR targets. However, an preceeding target to the same
MSHR, originally from a CPU, was serviced before the snoop, and caused
the block to be invalidated. This happens for ReadExReq and
UpgradeReq.

Note that the original snoop will receive a response, just not from
the cache in question, but instead from the cache upstream that issued
the ReadExReq or UpgradeReq.

Change-Id: I4ac012fbc8a46cf693ca390fe9476105d444e6f4
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>

mem: remove redudant check whether the cache forwards snoops

Change-Id: I57b56771086e1e2f512977fb7248d93c171ab925
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: change NULL to nullptr in the cache related classes

Change-Id: I5042410be54935650b7d05c84d8d9efbfcc06e70
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: fix the line length in the cache related classes

Change-Id: I6d1feb164a958dde0da87a1cd2698096112c4a82
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>

mem: Deallocate all write-queue entries when sent

This patch removes the write-queue entry tracking previously used for
uncacheable writes. The write-queue entry is now deallocated as soon
as the packet is sent. As a result we also forego the stats for
uncacheable writes. Additionally, there is no longer a need to attach
the write-queue entry to the packet.

mem: Align downstream cache packet creation in atomic and timing

This patch makes the control flow more uniform in atomic and timing,
ultimately making the code easier to understand.

mem: Add unused prefetch counter in caches

Added stat to the cache to account for HardPF'ed blocks that are evicted
before being referenced (over-prefetching).

mem: Create a separate class for the cache write buffer

This patch breaks out the cache write buffer into a separate class,
without affecting any stats. The goal of the patch is to avoid
encumbering the much-simpler write queue with the complex MSHR
handling. In a follow on patch this simplification allows us to
implement write combining.

The WriteQueue gets its own class, but shares a common ancestor, the
generic Queue, with the MSHRQueue.

mem: Ensure that InvalidateReq is not forwarded as ReadExReq

This patch fixes an issue where an InvalidationReq only traversed one
level of the cache hierarchy, and was subsequently turned into a
ReadExReq due to it needing writable, and the command not being
checked for explicitly.

mem: Move the point of coherency to the coherent crossbar

This patch introduces the ability of making the coherent crossbar the
point of coherency. If so, the crossbar does not forward packets where
a cache with ownership has already committed to responding, and also
does not forward any coherency-related packets that are not intended
for a downstream memory controller. Thus, invalidations and upgrades
are turned around in the crossbar, and the memory controller only sees
normal reads and writes.

In addition this patch moves the express snoop promotion of a packet
to the crossbar, thus allowing the downstream cache to check the
express snoop flag (as it should) for bypassing any blocking, rather
than relying on whether a cache is responding or not.

mem: Align cache behaviour in atomic when upstream is responding

Adopt the same flow as in timing mode, where the caches on the path to
memory get to keep the line (if present), and we use the
responderHadWritable flag to determine if we need to forward the
(invalidating) packet or not.

mem: Align how snoops are handled when hitting writebacks

This patch unifies the snoop handling in case of hitting writebacks
with how we handle snoops hitting in the tags. As a result, we end up
using the same optimisation as the normal snoops, where we inform the
downstream cache if we encounter a line in Modified (writable and
dirty) state, which enables us to avoid sending out express snoops to
invalidate any Shared copies of the line. A few regressions
consequently change, as some transactions are sunk higher up in the
cache hierarchy.

mem: add CacheVerbose debug flag, filter noisy DPRINTFs

Some of the DPRINTFs added to the classic cache in cset 45df88079f04,
while useful to those unfamiliar with the cache code, end up being
noise when you're familiar with the code but are trying to debug tricky
protocol issues. (Particularly getting two messages from each cache
as it receives a snoop request then declares that there was no match.)

This patch introduces a CacheVerbose debug flag, and moves a subset of
the added DPRINTFs into that category, so that Cache by itself returns
to being a more succinct summary of cache activity.

Also added a CacheAll compound flag to turn on all the cache-related
debug flags (other than CacheTags, which you *really* have to want badly
to turn it on, IMO).

mem: Do not allocate space for packet data if not needed

This patch looks at the request and response command to determine if
either actually has any data payload, and if not, we do not allocate
any space for packet data.

The only tricky case is where the command type is changed as part of
the MSHR functionality. In these cases where the original packet had
no data, but the new packet does, we need to explicitly call
allocate().

mem: Do not alter cache block state on uncacheable snoops

This patch ensures we do not respond with a Modified (dirty and
writable) line if the request is uncacheable, and that the cache
responding retains the line without modifying the state (even if
responding).

mem: Make cache terminology easier to understand

This patch changes the name of a bunch of packet flags and MSHR member
functions and variables to make the coherency protocol easier to
understand. In addition the patch adds and updates lots of
descriptions, explicitly spelling out assumptions.

The following name changes are made:

* the packet memInhibit flag is renamed to cacheResponding

* the packet sharedAsserted flag is renamed to hasSharers

* the packet NeedsExclusive attribute is renamed to NeedsWritable

* the packet isSupplyExclusive is renamed responderHadWritable

* the MSHR pendingDirty is renamed to pendingModified

The cache states, Modified, Owned, Exclusive, Shared are also called
out in the cache and MSHR code to make it easier to understand.

mem: Remove unused cache squash functionality

This patch removes the unused squash function from the MSHR queue, and
the associated (and also unused) threadNum member from the MSHR.

mem: Avoid unecessary checks when creating HardPFReq in cache

The checks made before sending out a HardPFReq were unecessarily
complex, and checked for cases that never occur. This patch
tidies it up.

mem: Do not use sender state to track forwarded snoops in cache

This patch changes how the cache tracks which snoops are forwarded,
and which ones are created locally. Previously the identification was
based on an empty sender state of a specific class, but this method
fails to distinguish which cache actually attached the sender
state. Instead we use the same mechanism as the crossbar, and keep
track of the requests that have outstanding snoops.

mem: Fix cache sender state handling and add clarification

This patch addresses a bug in how the cache attached the MSHR as a
sender state. Rather than overwriting any existing sender state it now
pushes a new one. The handling of upward snoops is also clarified.

mem: Fix memory allocation bug in deferred snoop handling

This patch fixes a corner case in the deferred snoop handling, where
requests ended up being used by multiple packets with different
lifetimes, and inadvertently got deleted while they were still in use.

mem: Add an option to perform clean writebacks from caches

This patch adds the necessary commands and cache functionality to
allow clean writebacks. This functionality is crucial, especially when
having exclusive (victim) caches. For example, if read-only L1
instruction caches are not sending clean writebacks, there will never
be any spills from the L1 to the L2. At the moment the cache model
defaults to not sending clean writebacks, and this should possibly be
re-evaluated.

The implementation of clean writebacks relies on a new packet command
WritebackClean, which acts much like a Writeback (renamed
WritebackDirty), and also much like a CleanEvict. On eviction of a
clean block the cache either sends a clean evict, or a clean
writeback, and if any copies are still cached upstream the clean
evict/writeback is dropped. Similarly, if a clean evict/writeback
reaches a cache where there are outstanding MSHRs for the block, the
packet is dropped. In the typical case though, the clean writeback
allocates a block in the downstream cache, and marks it writable if
the evicted block was writable.

The patch changes the O3_ARM_v7a L1 cache configuration and the
default L1 caches in config/common/Caches.py

mem: Add cache clusivity

This patch adds a parameter to control the cache clusivity, that is if
the cache is mostly inclusive or exclusive. At the moment there is no
intention to support strict policies, and thus the options are: 1)
mostly inclusive, or 2) mostly exclusive.

The choice of policy guides the behaviuor on a cache fill, and a new
helper function, allocOnFill, is created to encapsulate the decision
making process. For the timing mode, the decision is annotated on the
MSHR on sending out the downstream packet, and in atomic we directly
pass the decision to handleFill. We (ab)use the tempBlock in cases
where we are not allocating on fill, leaving the rest of the cache
unaffected. Simple and effective.

This patch also makes it more explicit that multiple caches are
allowed to consider a block writable (this is the case
also before this patch). That is, for a mostly inclusive cache,
multiple caches upstream may also consider the block exclusive. The
caches considering the block writable/exclusive all appear along the
same path to memory, and from a coherency protocol point of view it
works due to the fact that we always snoop upwards in zero time before
querying any downstream cache.

Note that this patch does not introduce clean writebacks. Thus, for
clean lines we are essentially removing a cache level if it is made
mostly exclusive. For example, lines from the read-only L1 instruction
cache or table-walker cache are always clean, and simply get dropped
rather than being passed to the L2. If the L2 is mostly exclusive and
does not allocate on fill it will thus never hold the line. A follow
on patch adds the clean writebacks.

The patch changes the L2 of the O3_ARM_v7a CPU configuration to be
mostly exclusive (and stats are affected accordingly).

mem: Enforce insertion order on the cache response path

This patch enforces insertion order transmission of packets on the
response path in the cache. Note that the logic to enforce order is
already present in the packet queue, this patch simply turns it on for
queues in the response path.

Without this patch, there are corner cases where a request-response is
faster than a response-response forwarded through the cache. This
violation of queuing order causes problems in the snoop filter leaving
it with inaccurate information. This causes assert failures in the
snoop filter later on.

A follow on patch relaxes the order enforcement in the packet queue to
limit the performance impact.

mem: Unify delayed packet deletion

This patch unifies how we deal with delayed packet deletion, where the
receiving slave is responsible for deleting the packet, but the
sending agent (e.g. a cache) is still relying on the pointer until the
call to sendTimingReq completes. Previously we used a mix of a
deletion vector and a construct using unique_ptr. With this patch we
ensure all slaves use the latter approach.

mem: Clarify cache MSHR handling on fill

This patch addresses the upgrading of deferred targets in the MSHR,
and makes it clearer by explicitly calling out what is happening
(deferred targets are promoted if we get exclusivity without asking
for it).

mem: Add check for block status on WriteLineReq fill

More checks to help with understanding of functionality.

mem: Fix WriteLineReq fill behaviour

This patch fixes issues in the interactions between deferred snoops
and WriteLineReq. More specifically, the patch addresses an issue
where deferred snoops caused assertion failures when being serviced on
the arrival of an InvalidateResp. The response packet was perceived to
be invalidating, when actually it is not for the cache that sent out
the original invalidation request.

mem: Add snoops for CleanEvicts and Writebacks in atomic mode

This patch mirrors the logic in timing mode which sends up snoops to
check for cached copies before sending CleanEvicts and Writebacks down
the memory hierarchy. In case there is a copy in a cache above,
discard CleanEvicts and set the BLOCK_CACHED flag in Writebacks so
that writebacks do not reset the cache residency bit in the snoop
filter below.

mem: Make the coherent crossbar account for timing snoops

This patch introduces the concept of a snoop latency. Given the
requirement to snoop and forward packets in zero time (due to the
coherency mechanism), the latency is accounted for later.

On a snoop, we establish the latency, and later add it to the header
delay of the packet. To allow multiple caches to contribute to the
snoop latency, we use a separate variable in the packet, and then take
the maximum before adding it to the header delay.

mem: Tidy up the snoop state-transition logic

Remove broken and unused option to pass dirty data on non-exclusive
snoops. Also beef up the comments a bit.

arm, mem: Remove unused CLEAR_LL request flag

Cleaning up dead code. The CLREX stores zero directly to
MISCREG_LOCKFLAG and so the request flag is no longer needed. The
corresponding functionality in the cache tags is also removed.

mem: Remove unused cache squash functionality

Tidying up.

mem: Add explicit Cache subclass and make BaseCache abstract

Open up for other subclasses to BaseCache and transition to using the
explicit Cache subclass.

mem: Move cache_impl.hh to cache.cc

There is no longer any need to keep the implementation in a header.

mem: Remove templates in cache model

This patch changes the cache implementation to rely on virtual methods
rather than using the replacement policy as a template argument.

There is no impact on the simulation performance, and overall the
changes make it easier to modify (and subclass) the cache and/or
replacement policy.

mem: refactor LRU cache tags and add random replacement tags

this patch implements a new tags class that uses a random replacement policy.
these tags prefer to evict invalid blocks first, if none are available a
replacement candidate is chosen at random.

this patch factors out the common code in the LRU class and creates a new
abstract class: the BaseSetAssoc class. any set associative tag class must
implement the functionality related to the actual replacement policy in the
following methods:

accessBlock()
findVictim()
insertBlock()
invalidate()

mem: Remove the cache builder

This patch removes the redundant cache builder class.

mem: Remove the IIC replacement policy

The IIC replacement policy seems to be unused and has probably
gathered too much bit rot to be useful. This patch removes the IIC and
its associated cache parameters.

remove the totally obsolete split cache

Get rid of coherence protocol object.

More major reorg of cache. Seems to work for atomic mode now,
timing mode still broken.

configs/example/memtest.py:
Revamp options.
src/cpu/memtest/memtest.cc:
No need for memory initialization.
No need to make atomic response... memory system should do that now.
src/cpu/memtest/memtest.hh:
MemTest really doesn't want to snoop.
src/mem/bridge.cc:
checkFunctional() cleanup.
src/mem/bus.cc:
src/mem/bus.hh:
src/mem/cache/base_cache.cc:
src/mem/cache/base_cache.hh:
src/mem/cache/cache.cc:
src/mem/cache/cache.hh:
src/mem/cache/cache_blk.hh:
src/mem/cache/cache_builder.cc:
src/mem/cache/cache_impl.hh:
src/mem/cache/coherence/coherence_protocol.cc:
src/mem/cache/coherence/coherence_protocol.hh:
src/mem/cache/coherence/simple_coherence.hh:
src/mem/cache/miss/SConscript:
src/mem/cache/miss/mshr.cc:
src/mem/cache/miss/mshr.hh:
src/mem/cache/miss/mshr_queue.cc:
src/mem/cache/miss/mshr_queue.hh:
src/mem/cache/prefetch/base_prefetcher.cc:
src/mem/cache/tags/fa_lru.cc:
src/mem/cache/tags/fa_lru.hh:
src/mem/cache/tags/iic.cc:
src/mem/cache/tags/iic.hh:
src/mem/cache/tags/lru.cc:
src/mem/cache/tags/lru.hh:
src/mem/cache/tags/split.cc:
src/mem/cache/tags/split.hh:
src/mem/cache/tags/split_lifo.cc:
src/mem/cache/tags/split_lifo.hh:
src/mem/cache/tags/split_lru.cc:
src/mem/cache/tags/split_lru.hh:
src/mem/packet.cc:
src/mem/packet.hh:
src/mem/physical.cc:
src/mem/physical.hh:
src/mem/tport.cc:
More major reorg. Seems to work for atomic mode now,
timing mode still broken.

First set of changes for reorganized cache coherence support.
Compiles but doesn't work... committing just so I can merge
(stupid bk!).

src/mem/bridge.cc:
Get rid of SNOOP_COMMIT.
src/mem/bus.cc:
src/mem/packet.hh:
Get rid of SNOOP_COMMIT & two-pass snoop.
First bits of EXPRESS_SNOOP support.
src/mem/cache/base_cache.cc:
src/mem/cache/base_cache.hh:
src/mem/cache/cache.hh:
src/mem/cache/cache_impl.hh:
src/mem/cache/miss/blocking_buffer.cc:
src/mem/cache/miss/miss_queue.cc:
src/mem/cache/prefetch/base_prefetcher.cc:
Big reorg of ports and port-related functions & events.
src/mem/cache/cache.cc:
src/mem/cache/cache_builder.cc:
src/mem/cache/coherence/SConscript:
Get rid of UniCoherence object.

Get rid of generic CacheTags object (fold back into Cache).

Turn cache MissQueue/BlockingBuffer into virtual object
instead of template parameter.

Make cache compression policy a runtime virtual thing
instead of a template policy.

Was having difficulty with merging the cache, reverted to an early version and will add back in the patches to make it work soon.

src/mem/cache/prefetch/tagged_prefetcher_impl.hh:
Trying to merge
src/mem/cache/base_cache.cc:
src/mem/cache/base_cache.hh:
src/mem/cache/cache.cc:
src/mem/cache/cache.hh:
src/mem/cache/cache_blk.hh:
src/mem/cache/cache_builder.cc:
src/mem/cache/cache_impl.hh:
src/mem/cache/coherence/coherence_protocol.cc:
src/mem/cache/coherence/coherence_protocol.hh:
src/mem/cache/coherence/simple_coherence.hh:
src/mem/cache/coherence/uni_coherence.cc:
src/mem/cache/coherence/uni_coherence.hh:
src/mem/cache/miss/blocking_buffer.cc:
src/mem/cache/miss/blocking_buffer.hh:
src/mem/cache/miss/miss_queue.cc:
src/mem/cache/miss/miss_queue.hh:
src/mem/cache/miss/mshr.cc:
src/mem/cache/miss/mshr.hh:
src/mem/cache/miss/mshr_queue.cc:
src/mem/cache/miss/mshr_queue.hh:
src/mem/cache/prefetch/base_prefetcher.cc:
src/mem/cache/prefetch/base_prefetcher.hh:
src/mem/cache/prefetch/ghb_prefetcher.cc:
src/mem/cache/prefetch/ghb_prefetcher.hh:
src/mem/cache/prefetch/stride_prefetcher.cc:
src/mem/cache/prefetch/stride_prefetcher.hh:
src/mem/cache/prefetch/tagged_prefetcher.hh:
src/mem/cache/tags/base_tags.cc:
src/mem/cache/tags/base_tags.hh:
src/mem/cache/tags/fa_lru.cc:
src/mem/cache/tags/fa_lru.hh:
src/mem/cache/tags/iic.cc:
src/mem/cache/tags/iic.hh:
src/mem/cache/tags/lru.cc:
src/mem/cache/tags/lru.hh:
src/mem/cache/tags/repl/gen.cc:
src/mem/cache/tags/repl/gen.hh:
src/mem/cache/tags/repl/repl.cc:
src/mem/cache/tags/repl/repl.hh:
src/mem/cache/tags/split.cc:
src/mem/cache/tags/split.hh:
src/mem/cache/tags/split_blk.hh:
src/mem/cache/tags/split_lifo.cc:
src/mem/cache/tags/split_lifo.hh:
src/mem/cache/tags/split_lru.cc:
src/mem/cache/tags/split_lru.hh:
Pulling an early version of the cache into the tree due to merging issues. Will apply patches and push.