111183Serfan.azarkhish@unibo.it# Copyright (c) 2012-2013 ARM Limited 211183Serfan.azarkhish@unibo.it# All rights reserved. 311183Serfan.azarkhish@unibo.it# 411183Serfan.azarkhish@unibo.it# The license below extends only to copyright in the software and shall 511183Serfan.azarkhish@unibo.it# not be construed as granting a license to any other intellectual 611183Serfan.azarkhish@unibo.it# property including but not limited to intellectual property relating 711183Serfan.azarkhish@unibo.it# to a hardware implementation of the functionality of the software 811183Serfan.azarkhish@unibo.it# licensed hereunder. You may use the software subject to the license 911183Serfan.azarkhish@unibo.it# terms below provided that you ensure that this notice is replicated 1011183Serfan.azarkhish@unibo.it# unmodified and in its entirety in all distributions of the software, 1111183Serfan.azarkhish@unibo.it# modified or unmodified, in source code or in binary form. 1211183Serfan.azarkhish@unibo.it# 1311183Serfan.azarkhish@unibo.it# Copyright (c) 2015 The University of Bologna 1411183Serfan.azarkhish@unibo.it# All rights reserved. 1511183Serfan.azarkhish@unibo.it# 1611183Serfan.azarkhish@unibo.it# Redistribution and use in source and binary forms, with or without 1711183Serfan.azarkhish@unibo.it# modification, are permitted provided that the following conditions are 1811183Serfan.azarkhish@unibo.it# met: redistributions of source code must retain the above copyright 1911183Serfan.azarkhish@unibo.it# notice, this list of conditions and the following disclaimer; 2011183Serfan.azarkhish@unibo.it# redistributions in binary form must reproduce the above copyright 2111183Serfan.azarkhish@unibo.it# notice, this list of conditions and the following disclaimer in the 2211183Serfan.azarkhish@unibo.it# documentation and/or other materials provided with the distribution; 2311183Serfan.azarkhish@unibo.it# neither the name of the copyright holders nor the names of its 2411183Serfan.azarkhish@unibo.it# contributors may be used to endorse or promote products derived from 2511183Serfan.azarkhish@unibo.it# this software without specific prior written permission. 2611183Serfan.azarkhish@unibo.it# 2711183Serfan.azarkhish@unibo.it# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 2811183Serfan.azarkhish@unibo.it# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 2911183Serfan.azarkhish@unibo.it# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 3011183Serfan.azarkhish@unibo.it# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 3111183Serfan.azarkhish@unibo.it# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 3211183Serfan.azarkhish@unibo.it# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 3311183Serfan.azarkhish@unibo.it# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 3411183Serfan.azarkhish@unibo.it# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 3511183Serfan.azarkhish@unibo.it# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 3611183Serfan.azarkhish@unibo.it# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 3711183Serfan.azarkhish@unibo.it# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 3811183Serfan.azarkhish@unibo.it# 3911183Serfan.azarkhish@unibo.it# Authors: Erfan Azarkhish 4011551Sabdul.mutaal@gmail.com# Abdul Mutaal Ahmad 4111183Serfan.azarkhish@unibo.it 4211183Serfan.azarkhish@unibo.it# A Simplified model of a complete HMC device. Based on: 4311183Serfan.azarkhish@unibo.it# [1] http://www.hybridmemorycube.org/specification-download/ 4411183Serfan.azarkhish@unibo.it# [2] High performance AXI-4.0 based interconnect for extensible smart memory 4511183Serfan.azarkhish@unibo.it# cubes(E. Azarkhish et. al) 4611183Serfan.azarkhish@unibo.it# [3] Low-Power Hybrid Memory Cubes With Link Power Management and Two-Level 4711183Serfan.azarkhish@unibo.it# Prefetching (J. Ahn et. al) 4811183Serfan.azarkhish@unibo.it# [4] Memory-centric system interconnect design with Hybrid Memory Cubes 4911183Serfan.azarkhish@unibo.it# (G. Kim et. al) 5011183Serfan.azarkhish@unibo.it# [5] Near Data Processing, Are we there yet? (M. Gokhale) 5111183Serfan.azarkhish@unibo.it# http://www.cs.utah.edu/wondp/gokhale.pdf 5211551Sabdul.mutaal@gmail.com# [6] openHMC - A Configurable Open-Source Hybrid Memory Cube Controller 5311551Sabdul.mutaal@gmail.com# (J. Schmidt) 5411551Sabdul.mutaal@gmail.com# [7] Hybrid Memory Cube performance characterization on data-centric 5511551Sabdul.mutaal@gmail.com# workloads (M. Gokhale) 5611183Serfan.azarkhish@unibo.it# 5711183Serfan.azarkhish@unibo.it# This script builds a complete HMC device composed of vault controllers, 5811183Serfan.azarkhish@unibo.it# serial links, the main internal crossbar, and an external hmc controller. 5911183Serfan.azarkhish@unibo.it# 6011183Serfan.azarkhish@unibo.it# - VAULT CONTROLLERS: 6111837Swendy.elsasser@arm.com# Instances of the HMC_2500_1x32 class with their functionality specified in 6211183Serfan.azarkhish@unibo.it# dram_ctrl.cc 6311183Serfan.azarkhish@unibo.it# 6411183Serfan.azarkhish@unibo.it# - THE MAIN XBAR: 6511183Serfan.azarkhish@unibo.it# This component is simply an instance of the NoncoherentXBar class, and its 6611183Serfan.azarkhish@unibo.it# parameters are tuned to [2]. 6711183Serfan.azarkhish@unibo.it# 6811551Sabdul.mutaal@gmail.com# - SERIAL LINKS CONTROLLER: 6911183Serfan.azarkhish@unibo.it# SerialLink is a simple variation of the Bridge class, with the ability to 7011551Sabdul.mutaal@gmail.com# account for the latency of packet serialization and controller latency. We 7111551Sabdul.mutaal@gmail.com# assume that the serializer component at the transmitter side does not need 7211551Sabdul.mutaal@gmail.com# to receive the whole packet to start the serialization. But the 7311551Sabdul.mutaal@gmail.com# deserializer waits for the complete packet to check its integrity first. 7411551Sabdul.mutaal@gmail.com# 7511183Serfan.azarkhish@unibo.it# * Bandwidth of the serial links is not modeled in the SerialLink component 7611551Sabdul.mutaal@gmail.com# itself. 7711183Serfan.azarkhish@unibo.it# 7811551Sabdul.mutaal@gmail.com# * Latency of serial link controller is composed of SerDes latency + link 7911551Sabdul.mutaal@gmail.com# controller 8011551Sabdul.mutaal@gmail.com# 8111183Serfan.azarkhish@unibo.it# * It is inferred from the standard [1] and the literature [3] that serial 8211183Serfan.azarkhish@unibo.it# links share the same address range and packets can travel over any of 8311183Serfan.azarkhish@unibo.it# them so a load distribution mechanism is required among them. 8411551Sabdul.mutaal@gmail.com# 8511551Sabdul.mutaal@gmail.com# ----------------------------------------- 8611551Sabdul.mutaal@gmail.com# | Host/HMC Controller | 8711551Sabdul.mutaal@gmail.com# | ---------------------- | 8811551Sabdul.mutaal@gmail.com# | | Link Aggregator | opt | 8911551Sabdul.mutaal@gmail.com# | ---------------------- | 9011551Sabdul.mutaal@gmail.com# | ---------------------- | 9111551Sabdul.mutaal@gmail.com# | | Serial Link + Ser | * 4 | 9211551Sabdul.mutaal@gmail.com# | ---------------------- | 9311551Sabdul.mutaal@gmail.com# |--------------------------------------- 9411551Sabdul.mutaal@gmail.com# ----------------------------------------- 9511551Sabdul.mutaal@gmail.com# | Device 9611551Sabdul.mutaal@gmail.com# | ---------------------- | 9711551Sabdul.mutaal@gmail.com# | | Xbar | * 4 | 9811551Sabdul.mutaal@gmail.com# | ---------------------- | 9911551Sabdul.mutaal@gmail.com# | ---------------------- | 10011551Sabdul.mutaal@gmail.com# | | Vault Controller | * 16 | 10111551Sabdul.mutaal@gmail.com# | ---------------------- | 10211551Sabdul.mutaal@gmail.com# | ---------------------- | 10311551Sabdul.mutaal@gmail.com# | | Memory | | 10411551Sabdul.mutaal@gmail.com# | ---------------------- | 10511551Sabdul.mutaal@gmail.com# |---------------------------------------| 10611551Sabdul.mutaal@gmail.com# 10711551Sabdul.mutaal@gmail.com# In this version we have present 3 different HMC archiecture along with 10811551Sabdul.mutaal@gmail.com# alongwith their corresponding test script. 10911551Sabdul.mutaal@gmail.com# 11011551Sabdul.mutaal@gmail.com# same: It has 4 crossbars in HMC memory. All the crossbars are connected 11111551Sabdul.mutaal@gmail.com# to each other, providing complete memory range. This archicture also covers 11211551Sabdul.mutaal@gmail.com# the added latency for sending a request to non-local vault(bridge in b/t 11311551Sabdul.mutaal@gmail.com# crossbars). All the 4 serial links can access complete memory. So each 11411551Sabdul.mutaal@gmail.com# link can be connected to separate processor. 11511551Sabdul.mutaal@gmail.com# 11611551Sabdul.mutaal@gmail.com# distributed: It has 4 crossbars inside the HMC. Crossbars are not 11711551Sabdul.mutaal@gmail.com# connected.Through each crossbar only local vaults can be accessed. But to 11811551Sabdul.mutaal@gmail.com# support this architecture we need a crossbar between serial links and 11911551Sabdul.mutaal@gmail.com# processor. 12011551Sabdul.mutaal@gmail.com# 12111551Sabdul.mutaal@gmail.com# mixed: This is a hybrid architecture. It has 4 crossbars inside the HMC. 12211551Sabdul.mutaal@gmail.com# 2 Crossbars are connected to only local vaults. From other 2 crossbar, a 12311551Sabdul.mutaal@gmail.com# request can be forwarded to any other vault. 12411183Serfan.azarkhish@unibo.it 12513774Sandreas.sandberg@arm.comfrom __future__ import print_function 12613774Sandreas.sandberg@arm.comfrom __future__ import absolute_import 12713774Sandreas.sandberg@arm.com 12812340Szulian@eit.uni-kl.deimport argparse 12911183Serfan.azarkhish@unibo.it 13011183Serfan.azarkhish@unibo.itimport m5 13111183Serfan.azarkhish@unibo.itfrom m5.objects import * 13212340Szulian@eit.uni-kl.defrom m5.util import * 13311183Serfan.azarkhish@unibo.it 13412340Szulian@eit.uni-kl.de 13512340Szulian@eit.uni-kl.dedef add_options(parser): 13612340Szulian@eit.uni-kl.de # *****************************CROSSBAR PARAMETERS************************* 13711183Serfan.azarkhish@unibo.it # Flit size of the main interconnect [1] 13812340Szulian@eit.uni-kl.de parser.add_argument("--xbar-width", default=32, action="store", type=int, 13912340Szulian@eit.uni-kl.de help="Data width of the main XBar (Bytes)") 14011183Serfan.azarkhish@unibo.it 14111183Serfan.azarkhish@unibo.it # Clock frequency of the main interconnect [1] 14211183Serfan.azarkhish@unibo.it # This crossbar, is placed on the logic-based of the HMC and it has its 14311183Serfan.azarkhish@unibo.it # own voltage and clock domains, different from the DRAM dies or from the 14411183Serfan.azarkhish@unibo.it # host. 14512340Szulian@eit.uni-kl.de parser.add_argument("--xbar-frequency", default='1GHz', type=str, 14612340Szulian@eit.uni-kl.de help="Clock Frequency of the main XBar") 14711183Serfan.azarkhish@unibo.it 14811183Serfan.azarkhish@unibo.it # Arbitration latency of the HMC XBar [1] 14912340Szulian@eit.uni-kl.de parser.add_argument("--xbar-frontend-latency", default=1, action="store", 15012340Szulian@eit.uni-kl.de type=int, help="Arbitration latency of the XBar") 15111183Serfan.azarkhish@unibo.it 15211183Serfan.azarkhish@unibo.it # Latency to forward a packet via the interconnect [1](two levels of FIFOs 15311183Serfan.azarkhish@unibo.it # at the input and output of the inteconnect) 15412340Szulian@eit.uni-kl.de parser.add_argument("--xbar-forward-latency", default=2, action="store", 15512340Szulian@eit.uni-kl.de type=int, help="Forward latency of the XBar") 15611183Serfan.azarkhish@unibo.it 15711183Serfan.azarkhish@unibo.it # Latency to forward a response via the interconnect [1](two levels of 15811183Serfan.azarkhish@unibo.it # FIFOs at the input and output of the inteconnect) 15912340Szulian@eit.uni-kl.de parser.add_argument("--xbar-response-latency", default=2, action="store", 16012340Szulian@eit.uni-kl.de type=int, help="Response latency of the XBar") 16111183Serfan.azarkhish@unibo.it 16211551Sabdul.mutaal@gmail.com # number of cross which connects 16 Vaults to serial link[7] 16312340Szulian@eit.uni-kl.de parser.add_argument("--number-mem-crossbar", default=4, action="store", 16412340Szulian@eit.uni-kl.de type=int, help="Number of crossbar in HMC") 16511551Sabdul.mutaal@gmail.com 16612340Szulian@eit.uni-kl.de # *****************************SERIAL LINK PARAMETERS********************** 16711551Sabdul.mutaal@gmail.com # Number of serial links controllers [1] 16812340Szulian@eit.uni-kl.de parser.add_argument("--num-links-controllers", default=4, action="store", 16912340Szulian@eit.uni-kl.de type=int, help="Number of serial links") 17011183Serfan.azarkhish@unibo.it 17111183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the request side of the serial 17211183Serfan.azarkhish@unibo.it # link. This number should be adjusted to achive required bandwidth 17312340Szulian@eit.uni-kl.de parser.add_argument("--link-buffer-size-req", default=10, action="store", 17412340Szulian@eit.uni-kl.de type=int, help="Number of packets to buffer at the\ 17512340Szulian@eit.uni-kl.de request side of the serial link") 17611183Serfan.azarkhish@unibo.it 17711183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the response side of the serial 17811183Serfan.azarkhish@unibo.it # link. This number should be adjusted to achive required bandwidth 17912340Szulian@eit.uni-kl.de parser.add_argument("--link-buffer-size-rsp", default=10, action="store", 18012340Szulian@eit.uni-kl.de type=int, help="Number of packets to buffer at the\ 18112340Szulian@eit.uni-kl.de response side of the serial link") 18211183Serfan.azarkhish@unibo.it 18311183Serfan.azarkhish@unibo.it # Latency of the serial link composed by SER/DES latency (1.6ns [4]) plus 18411183Serfan.azarkhish@unibo.it # the PCB trace latency (3ns Estimated based on [5]) 18512340Szulian@eit.uni-kl.de parser.add_argument("--link-latency", default='4.6ns', type=str, 18612340Szulian@eit.uni-kl.de help="Latency of the serial links") 18711183Serfan.azarkhish@unibo.it 18811551Sabdul.mutaal@gmail.com # Clock frequency of the each serial link(SerDes) [1] 18912340Szulian@eit.uni-kl.de parser.add_argument("--link-frequency", default='10GHz', type=str, 19012340Szulian@eit.uni-kl.de help="Clock Frequency of the serial links") 19111183Serfan.azarkhish@unibo.it 19211551Sabdul.mutaal@gmail.com # Clock frequency of serial link Controller[6] 19311551Sabdul.mutaal@gmail.com # clk_hmc[Mhz]= num_lanes_per_link * lane_speed [Gbits/s] / 19411551Sabdul.mutaal@gmail.com # data_path_width * 10^6 19511551Sabdul.mutaal@gmail.com # clk_hmc[Mhz]= 16 * 10 Gbps / 256 * 10^6 = 625 Mhz 19612340Szulian@eit.uni-kl.de parser.add_argument("--link-controller-frequency", default='625MHz', 19712340Szulian@eit.uni-kl.de type=str, help="Clock Frequency of the link\ 19812340Szulian@eit.uni-kl.de controller") 19911551Sabdul.mutaal@gmail.com 20011551Sabdul.mutaal@gmail.com # Latency of the serial link controller to process the packets[1][6] 20111551Sabdul.mutaal@gmail.com # (ClockDomain = 625 Mhz ) 20211551Sabdul.mutaal@gmail.com # used here for calculations only 20312340Szulian@eit.uni-kl.de parser.add_argument("--link-ctrl-latency", default=4, action="store", 20412340Szulian@eit.uni-kl.de type=int, help="The number of cycles required for the\ 20512340Szulian@eit.uni-kl.de controller to process the packet") 20611551Sabdul.mutaal@gmail.com 20711551Sabdul.mutaal@gmail.com # total_ctrl_latency = link_ctrl_latency + link_latency 20811551Sabdul.mutaal@gmail.com # total_ctrl_latency = 4(Cycles) * 1.6 ns + 4.6 ns 20912340Szulian@eit.uni-kl.de parser.add_argument("--total-ctrl-latency", default='11ns', type=str, 21012340Szulian@eit.uni-kl.de help="The latency experienced by every packet\ 21112340Szulian@eit.uni-kl.de regardless of size of packet") 21211551Sabdul.mutaal@gmail.com 21311183Serfan.azarkhish@unibo.it # Number of parallel lanes in each serial link [1] 21412340Szulian@eit.uni-kl.de parser.add_argument("--num-lanes-per-link", default=16, action="store", 21512340Szulian@eit.uni-kl.de type=int, help="Number of lanes per each link") 21611183Serfan.azarkhish@unibo.it 21711183Serfan.azarkhish@unibo.it # Number of serial links [1] 21812340Szulian@eit.uni-kl.de parser.add_argument("--num-serial-links", default=4, action="store", 21912340Szulian@eit.uni-kl.de type=int, help="Number of serial links") 22011183Serfan.azarkhish@unibo.it 22111551Sabdul.mutaal@gmail.com # speed of each lane of serial link - SerDes serial interface 10 Gb/s 22212340Szulian@eit.uni-kl.de parser.add_argument("--serial-link-speed", default=10, action="store", 22312340Szulian@eit.uni-kl.de type=int, help="Gbs/s speed of each lane of serial\ 22412340Szulian@eit.uni-kl.de link") 22511183Serfan.azarkhish@unibo.it 22612340Szulian@eit.uni-kl.de # address range for each of the serial links 22712340Szulian@eit.uni-kl.de parser.add_argument("--serial-link-addr-range", default='1GB', type=str, 22812340Szulian@eit.uni-kl.de help="memory range for each of the serial links.\ 22912340Szulian@eit.uni-kl.de Default: 1GB") 23012340Szulian@eit.uni-kl.de 23112340Szulian@eit.uni-kl.de # *****************************PERFORMANCE MONITORING********************* 23211183Serfan.azarkhish@unibo.it # The main monitor behind the HMC Controller 23312340Szulian@eit.uni-kl.de parser.add_argument("--enable-global-monitor", action="store_true", 23412340Szulian@eit.uni-kl.de help="The main monitor behind the HMC Controller") 23511183Serfan.azarkhish@unibo.it 23611183Serfan.azarkhish@unibo.it # The link performance monitors 23712340Szulian@eit.uni-kl.de parser.add_argument("--enable-link-monitor", action="store_true", 23812340Szulian@eit.uni-kl.de help="The link monitors") 23911183Serfan.azarkhish@unibo.it 24011551Sabdul.mutaal@gmail.com # link aggregator enable - put a cross between buffers & links 24112340Szulian@eit.uni-kl.de parser.add_argument("--enable-link-aggr", action="store_true", help="The\ 24212340Szulian@eit.uni-kl.de crossbar between port and Link Controller") 24311183Serfan.azarkhish@unibo.it 24412340Szulian@eit.uni-kl.de parser.add_argument("--enable-buff-div", action="store_true", 24512340Szulian@eit.uni-kl.de help="Memory Range of Buffer is ivided between total\ 24612340Szulian@eit.uni-kl.de range") 24711183Serfan.azarkhish@unibo.it 24812340Szulian@eit.uni-kl.de # *****************************HMC ARCHITECTURE ************************** 24911551Sabdul.mutaal@gmail.com # Memory chunk for 16 vault - numbers of vault / number of crossbars 25012340Szulian@eit.uni-kl.de parser.add_argument("--mem-chunk", default=4, action="store", type=int, 25112340Szulian@eit.uni-kl.de help="Chunk of memory range for each cross bar in\ 25212340Szulian@eit.uni-kl.de arch 0") 25311551Sabdul.mutaal@gmail.com 25411551Sabdul.mutaal@gmail.com # size of req buffer within crossbar, used for modelling extra latency 25511551Sabdul.mutaal@gmail.com # when the reuqest go to non-local vault 25612340Szulian@eit.uni-kl.de parser.add_argument("--xbar-buffer-size-req", default=10, action="store", 25712340Szulian@eit.uni-kl.de type=int, help="Number of packets to buffer at the\ 25812340Szulian@eit.uni-kl.de request side of the crossbar") 25911551Sabdul.mutaal@gmail.com 26011551Sabdul.mutaal@gmail.com # size of response buffer within crossbar, used for modelling extra latency 26111551Sabdul.mutaal@gmail.com # when the response received from non-local vault 26212340Szulian@eit.uni-kl.de parser.add_argument("--xbar-buffer-size-resp", default=10, action="store", 26312340Szulian@eit.uni-kl.de type=int, help="Number of packets to buffer at the\ 26412340Szulian@eit.uni-kl.de response side of the crossbar") 26512340Szulian@eit.uni-kl.de # HMC device architecture. It affects the HMC host controller as well 26612340Szulian@eit.uni-kl.de parser.add_argument("--arch", type=str, choices=["same", "distributed", 26712340Szulian@eit.uni-kl.de "mixed"], default="distributed", help="same: HMC with\ 26812340Szulian@eit.uni-kl.de 4 links, all with same range.\ndistributed: HMC with\ 26912340Szulian@eit.uni-kl.de 4 links with distributed range.\nmixed: mixed with\ 27012340Szulian@eit.uni-kl.de same and distributed range.\nDefault: distributed") 27112340Szulian@eit.uni-kl.de # HMC device - number of vaults 27212340Szulian@eit.uni-kl.de parser.add_argument("--hmc-dev-num-vaults", default=16, action="store", 27312340Szulian@eit.uni-kl.de type=int, help="number of independent vaults within\ 27412340Szulian@eit.uni-kl.de the HMC device. Note: each vault has a memory\ 27512340Szulian@eit.uni-kl.de controller (valut controller)\nDefault: 16") 27612340Szulian@eit.uni-kl.de # HMC device - vault capacity or size 27712340Szulian@eit.uni-kl.de parser.add_argument("--hmc-dev-vault-size", default='256MB', type=str, 27812340Szulian@eit.uni-kl.de help="vault storage capacity in bytes. Default:\ 27912340Szulian@eit.uni-kl.de 256MB") 28012340Szulian@eit.uni-kl.de parser.add_argument("--mem-type", type=str, choices=["HMC_2500_1x32"], 28112340Szulian@eit.uni-kl.de default="HMC_2500_1x32", help="type of HMC memory to\ 28212340Szulian@eit.uni-kl.de use. Default: HMC_2500_1x32") 28312340Szulian@eit.uni-kl.de parser.add_argument("--mem-channels", default=1, action="store", type=int, 28412340Szulian@eit.uni-kl.de help="Number of memory channels") 28512340Szulian@eit.uni-kl.de parser.add_argument("--mem-ranks", default=1, action="store", type=int, 28612340Szulian@eit.uni-kl.de help="Number of ranks to iterate across") 28712340Szulian@eit.uni-kl.de parser.add_argument("--burst-length", default=256, action="store", 28812340Szulian@eit.uni-kl.de type=int, help="burst length in bytes. Note: the\ 28912340Szulian@eit.uni-kl.de cache line size will be set to this value.\nDefault:\ 29012340Szulian@eit.uni-kl.de 256") 29111551Sabdul.mutaal@gmail.com 29211551Sabdul.mutaal@gmail.com 29312340Szulian@eit.uni-kl.de# configure HMC host controller 29412340Szulian@eit.uni-kl.dedef config_hmc_host_ctrl(opt, system): 29511551Sabdul.mutaal@gmail.com 29612340Szulian@eit.uni-kl.de # create HMC host controller 29712340Szulian@eit.uni-kl.de system.hmc_host = SubSystem() 29811183Serfan.azarkhish@unibo.it 29912340Szulian@eit.uni-kl.de # Create additional crossbar for arch1 30012340Szulian@eit.uni-kl.de if opt.arch == "distributed" or opt.arch == "mixed": 30112340Szulian@eit.uni-kl.de clk = '100GHz' 30212340Szulian@eit.uni-kl.de vd = VoltageDomain(voltage='1V') 30312340Szulian@eit.uni-kl.de # Create additional crossbar for arch1 30412340Szulian@eit.uni-kl.de system.membus = NoncoherentXBar(width=8) 30512340Szulian@eit.uni-kl.de system.membus.badaddr_responder = BadAddr() 30612340Szulian@eit.uni-kl.de system.membus.default = Self.badaddr_responder.pio 30712340Szulian@eit.uni-kl.de system.membus.width = 8 30812340Szulian@eit.uni-kl.de system.membus.frontend_latency = 3 30912340Szulian@eit.uni-kl.de system.membus.forward_latency = 4 31012340Szulian@eit.uni-kl.de system.membus.response_latency = 2 31112340Szulian@eit.uni-kl.de cd = SrcClockDomain(clock=clk, voltage_domain=vd) 31212340Szulian@eit.uni-kl.de system.membus.clk_domain = cd 31311183Serfan.azarkhish@unibo.it 31412340Szulian@eit.uni-kl.de # create memory ranges for the serial links 31512340Szulian@eit.uni-kl.de slar = convert.toMemorySize(opt.serial_link_addr_range) 31612340Szulian@eit.uni-kl.de # Memmory ranges of serial link for arch-0. Same as the ranges of vault 31712340Szulian@eit.uni-kl.de # controllers (4 vaults to 1 serial link) 31812340Szulian@eit.uni-kl.de if opt.arch == "same": 31912340Szulian@eit.uni-kl.de ser_ranges = [AddrRange(0, (4*slar)-1) for i in 32012340Szulian@eit.uni-kl.de range(opt.num_serial_links)] 32112340Szulian@eit.uni-kl.de # Memmory ranges of serial link for arch-1. Distributed range accross 32212340Szulian@eit.uni-kl.de # links 32312340Szulian@eit.uni-kl.de if opt.arch == "distributed": 32412340Szulian@eit.uni-kl.de ser_ranges = [AddrRange(i*slar, ((i+1)*slar)-1) for i in 32512340Szulian@eit.uni-kl.de range(opt.num_serial_links)] 32612340Szulian@eit.uni-kl.de # Memmory ranges of serial link for arch-2 'Mixed' address distribution 32712340Szulian@eit.uni-kl.de # over links 32812340Szulian@eit.uni-kl.de if opt.arch == "mixed": 32912340Szulian@eit.uni-kl.de ser_range0 = AddrRange(0, (1*slar)-1) 33012340Szulian@eit.uni-kl.de ser_range1 = AddrRange(1*slar, 2*slar-1) 33112340Szulian@eit.uni-kl.de ser_range2 = AddrRange(0, (4*slar)-1) 33212340Szulian@eit.uni-kl.de ser_range3 = AddrRange(0, (4*slar)-1) 33312340Szulian@eit.uni-kl.de ser_ranges = [ser_range0, ser_range1, ser_range2, ser_range3] 33412340Szulian@eit.uni-kl.de 33512340Szulian@eit.uni-kl.de # Serial link Controller with 16 SerDes links at 10 Gbps with serial link 33612340Szulian@eit.uni-kl.de # ranges w.r.t to architecture 33712340Szulian@eit.uni-kl.de sl = [SerialLink(ranges=ser_ranges[i], 33812340Szulian@eit.uni-kl.de req_size=opt.link_buffer_size_req, 33912340Szulian@eit.uni-kl.de resp_size=opt.link_buffer_size_rsp, 34012340Szulian@eit.uni-kl.de num_lanes=opt.num_lanes_per_link, 34112340Szulian@eit.uni-kl.de link_speed=opt.serial_link_speed, 34212340Szulian@eit.uni-kl.de delay=opt.total_ctrl_latency) for i in 34313731Sandreas.sandberg@arm.com range(opt.num_serial_links)] 34412340Szulian@eit.uni-kl.de system.hmc_host.seriallink = sl 34511183Serfan.azarkhish@unibo.it 34611551Sabdul.mutaal@gmail.com # enable global monitor 34712340Szulian@eit.uni-kl.de if opt.enable_global_monitor: 34812340Szulian@eit.uni-kl.de system.hmc_host.lmonitor = [CommMonitor() for i in 34913731Sandreas.sandberg@arm.com range(opt.num_serial_links)] 35011183Serfan.azarkhish@unibo.it 35111551Sabdul.mutaal@gmail.com # set the clock frequency for serial link 35213731Sandreas.sandberg@arm.com for i in range(opt.num_serial_links): 35312340Szulian@eit.uni-kl.de clk = opt.link_controller_frequency 35412340Szulian@eit.uni-kl.de vd = VoltageDomain(voltage='1V') 35512340Szulian@eit.uni-kl.de scd = SrcClockDomain(clock=clk, voltage_domain=vd) 35612340Szulian@eit.uni-kl.de system.hmc_host.seriallink[i].clk_domain = scd 35711183Serfan.azarkhish@unibo.it 35811551Sabdul.mutaal@gmail.com # Connect membus/traffic gen to Serial Link Controller for differrent HMC 35911551Sabdul.mutaal@gmail.com # architectures 36012340Szulian@eit.uni-kl.de hh = system.hmc_host 36112340Szulian@eit.uni-kl.de if opt.arch == "distributed": 36212340Szulian@eit.uni-kl.de mb = system.membus 36313731Sandreas.sandberg@arm.com for i in range(opt.num_links_controllers): 36412340Szulian@eit.uni-kl.de if opt.enable_global_monitor: 36512340Szulian@eit.uni-kl.de mb.master = hh.lmonitor[i].slave 36612340Szulian@eit.uni-kl.de hh.lmonitor[i].master = hh.seriallink[i].slave 36711551Sabdul.mutaal@gmail.com else: 36812340Szulian@eit.uni-kl.de mb.master = hh.seriallink[i].slave 36912340Szulian@eit.uni-kl.de if opt.arch == "mixed": 37012340Szulian@eit.uni-kl.de mb = system.membus 37112340Szulian@eit.uni-kl.de if opt.enable_global_monitor: 37212340Szulian@eit.uni-kl.de mb.master = hh.lmonitor[0].slave 37312340Szulian@eit.uni-kl.de hh.lmonitor[0].master = hh.seriallink[0].slave 37412340Szulian@eit.uni-kl.de mb.master = hh.lmonitor[1].slave 37512340Szulian@eit.uni-kl.de hh.lmonitor[1].master = hh.seriallink[1].slave 37612340Szulian@eit.uni-kl.de else: 37712340Szulian@eit.uni-kl.de mb.master = hh.seriallink[0].slave 37812340Szulian@eit.uni-kl.de mb.master = hh.seriallink[1].slave 37911183Serfan.azarkhish@unibo.it 38012340Szulian@eit.uni-kl.de if opt.arch == "same": 38113731Sandreas.sandberg@arm.com for i in range(opt.num_links_controllers): 38212340Szulian@eit.uni-kl.de if opt.enable_global_monitor: 38312340Szulian@eit.uni-kl.de hh.lmonitor[i].master = hh.seriallink[i].slave 38411183Serfan.azarkhish@unibo.it 38511551Sabdul.mutaal@gmail.com return system 38611551Sabdul.mutaal@gmail.com 38711551Sabdul.mutaal@gmail.com 38812340Szulian@eit.uni-kl.de# Create an HMC device 38912340Szulian@eit.uni-kl.dedef config_hmc_dev(opt, system, hmc_host): 39011551Sabdul.mutaal@gmail.com 39112340Szulian@eit.uni-kl.de # create HMC device 39212340Szulian@eit.uni-kl.de system.hmc_dev = SubSystem() 39311551Sabdul.mutaal@gmail.com 39412340Szulian@eit.uni-kl.de # create memory ranges for the vault controllers 39512340Szulian@eit.uni-kl.de arv = convert.toMemorySize(opt.hmc_dev_vault_size) 39612340Szulian@eit.uni-kl.de addr_ranges_vaults = [AddrRange(i*arv, ((i+1)*arv-1)) for i in 39712340Szulian@eit.uni-kl.de range(opt.hmc_dev_num_vaults)] 39812340Szulian@eit.uni-kl.de system.mem_ranges = addr_ranges_vaults 39911551Sabdul.mutaal@gmail.com 40012340Szulian@eit.uni-kl.de if opt.enable_link_monitor: 40113731Sandreas.sandberg@arm.com lm = [CommMonitor() for i in range(opt.num_links_controllers)] 40212340Szulian@eit.uni-kl.de system.hmc_dev.lmonitor = lm 40311551Sabdul.mutaal@gmail.com 40411551Sabdul.mutaal@gmail.com # 4 HMC Crossbars located in its logic-base (LoB) 40512340Szulian@eit.uni-kl.de xb = [NoncoherentXBar(width=opt.xbar_width, 40612340Szulian@eit.uni-kl.de frontend_latency=opt.xbar_frontend_latency, 40712340Szulian@eit.uni-kl.de forward_latency=opt.xbar_forward_latency, 40812340Szulian@eit.uni-kl.de response_latency=opt.xbar_response_latency) for i in 40913731Sandreas.sandberg@arm.com range(opt.number_mem_crossbar)] 41012340Szulian@eit.uni-kl.de system.hmc_dev.xbar = xb 41111551Sabdul.mutaal@gmail.com 41213731Sandreas.sandberg@arm.com for i in range(opt.number_mem_crossbar): 41312340Szulian@eit.uni-kl.de clk = opt.xbar_frequency 41412340Szulian@eit.uni-kl.de vd = VoltageDomain(voltage='1V') 41512340Szulian@eit.uni-kl.de scd = SrcClockDomain(clock=clk, voltage_domain=vd) 41612340Szulian@eit.uni-kl.de system.hmc_dev.xbar[i].clk_domain = scd 41711551Sabdul.mutaal@gmail.com 41811551Sabdul.mutaal@gmail.com # Attach 4 serial link to 4 crossbar/s 41913731Sandreas.sandberg@arm.com for i in range(opt.num_serial_links): 42012340Szulian@eit.uni-kl.de if opt.enable_link_monitor: 42111551Sabdul.mutaal@gmail.com system.hmc_host.seriallink[i].master = \ 42211551Sabdul.mutaal@gmail.com system.hmc_dev.lmonitor[i].slave 42311551Sabdul.mutaal@gmail.com system.hmc_dev.lmonitor[i].master = system.hmc_dev.xbar[i].slave 42411183Serfan.azarkhish@unibo.it else: 42511551Sabdul.mutaal@gmail.com system.hmc_host.seriallink[i].master = system.hmc_dev.xbar[i].slave 42611551Sabdul.mutaal@gmail.com 42711551Sabdul.mutaal@gmail.com # Connecting xbar with each other for request arriving at the wrong xbar, 42811551Sabdul.mutaal@gmail.com # then it will be forward to correct xbar. Bridge is used to connect xbars 42912340Szulian@eit.uni-kl.de if opt.arch == "same": 43011551Sabdul.mutaal@gmail.com numx = len(system.hmc_dev.xbar) 43111551Sabdul.mutaal@gmail.com 43211551Sabdul.mutaal@gmail.com # create a list of buffers 43312340Szulian@eit.uni-kl.de system.hmc_dev.buffers = [Bridge(req_size=opt.xbar_buffer_size_req, 43412340Szulian@eit.uni-kl.de resp_size=opt.xbar_buffer_size_resp) 43513731Sandreas.sandberg@arm.com for i in range(numx*(opt.mem_chunk-1))] 43611551Sabdul.mutaal@gmail.com 43711551Sabdul.mutaal@gmail.com # Buffer iterator 43811551Sabdul.mutaal@gmail.com it = iter(range(len(system.hmc_dev.buffers))) 43911551Sabdul.mutaal@gmail.com 44011551Sabdul.mutaal@gmail.com # necesarry to add system_port to one of the xbar 44111551Sabdul.mutaal@gmail.com system.system_port = system.hmc_dev.xbar[3].slave 44211551Sabdul.mutaal@gmail.com 44311551Sabdul.mutaal@gmail.com # iterate over all the crossbars and connect them as required 44411551Sabdul.mutaal@gmail.com for i in range(numx): 44511551Sabdul.mutaal@gmail.com for j in range(numx): 44611551Sabdul.mutaal@gmail.com # connect xbar to all other xbars except itself 44711551Sabdul.mutaal@gmail.com if i != j: 44811551Sabdul.mutaal@gmail.com # get the next index of buffer 44911551Sabdul.mutaal@gmail.com index = it.next() 45011551Sabdul.mutaal@gmail.com 45111551Sabdul.mutaal@gmail.com # Change the default values for ranges of bridge 45211551Sabdul.mutaal@gmail.com system.hmc_dev.buffers[index].ranges = system.mem_ranges[ 45312340Szulian@eit.uni-kl.de j * int(opt.mem_chunk): 45412340Szulian@eit.uni-kl.de (j + 1) * int(opt.mem_chunk)] 45511551Sabdul.mutaal@gmail.com 45611551Sabdul.mutaal@gmail.com # Connect the bridge between corssbars 45711551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[i].master = system.hmc_dev.buffers[ 45811551Sabdul.mutaal@gmail.com index].slave 45911551Sabdul.mutaal@gmail.com system.hmc_dev.buffers[ 46011551Sabdul.mutaal@gmail.com index].master = system.hmc_dev.xbar[j].slave 46111551Sabdul.mutaal@gmail.com else: 46211551Sabdul.mutaal@gmail.com # Don't connect the xbar to itself 46311551Sabdul.mutaal@gmail.com pass 46411551Sabdul.mutaal@gmail.com 46511551Sabdul.mutaal@gmail.com # Two crossbars are connected to all other crossbars-Other 2 vault 46611551Sabdul.mutaal@gmail.com # can only direct traffic to it local vaults 46712340Szulian@eit.uni-kl.de if opt.arch == "mixed": 46811551Sabdul.mutaal@gmail.com system.hmc_dev.buffer30 = Bridge(ranges=system.mem_ranges[0:4]) 46911551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[3].master = system.hmc_dev.buffer30.slave 47011551Sabdul.mutaal@gmail.com system.hmc_dev.buffer30.master = system.hmc_dev.xbar[0].slave 47111551Sabdul.mutaal@gmail.com 47211551Sabdul.mutaal@gmail.com system.hmc_dev.buffer31 = Bridge(ranges=system.mem_ranges[4:8]) 47311551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[3].master = system.hmc_dev.buffer31.slave 47411551Sabdul.mutaal@gmail.com system.hmc_dev.buffer31.master = system.hmc_dev.xbar[1].slave 47511551Sabdul.mutaal@gmail.com 47611551Sabdul.mutaal@gmail.com system.hmc_dev.buffer32 = Bridge(ranges=system.mem_ranges[8:12]) 47711551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[3].master = system.hmc_dev.buffer32.slave 47811551Sabdul.mutaal@gmail.com system.hmc_dev.buffer32.master = system.hmc_dev.xbar[2].slave 47911551Sabdul.mutaal@gmail.com 48011551Sabdul.mutaal@gmail.com system.hmc_dev.buffer20 = Bridge(ranges=system.mem_ranges[0:4]) 48111551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[2].master = system.hmc_dev.buffer20.slave 48211551Sabdul.mutaal@gmail.com system.hmc_dev.buffer20.master = system.hmc_dev.xbar[0].slave 48311551Sabdul.mutaal@gmail.com 48411551Sabdul.mutaal@gmail.com system.hmc_dev.buffer21 = Bridge(ranges=system.mem_ranges[4:8]) 48511551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[2].master = system.hmc_dev.buffer21.slave 48611551Sabdul.mutaal@gmail.com system.hmc_dev.buffer21.master = system.hmc_dev.xbar[1].slave 48711551Sabdul.mutaal@gmail.com 48811551Sabdul.mutaal@gmail.com system.hmc_dev.buffer23 = Bridge(ranges=system.mem_ranges[12:16]) 48911551Sabdul.mutaal@gmail.com system.hmc_dev.xbar[2].master = system.hmc_dev.buffer23.slave 49011551Sabdul.mutaal@gmail.com system.hmc_dev.buffer23.master = system.hmc_dev.xbar[3].slave 491