HMC.py revision 11551
13569Sgblack@eecs.umich.edu# Copyright (c) 2012-2013 ARM Limited 23569Sgblack@eecs.umich.edu# All rights reserved. 33569Sgblack@eecs.umich.edu# 43569Sgblack@eecs.umich.edu# The license below extends only to copyright in the software and shall 53569Sgblack@eecs.umich.edu# not be construed as granting a license to any other intellectual 63569Sgblack@eecs.umich.edu# property including but not limited to intellectual property relating 73569Sgblack@eecs.umich.edu# to a hardware implementation of the functionality of the software 83569Sgblack@eecs.umich.edu# licensed hereunder. You may use the software subject to the license 93569Sgblack@eecs.umich.edu# terms below provided that you ensure that this notice is replicated 103569Sgblack@eecs.umich.edu# unmodified and in its entirety in all distributions of the software, 113569Sgblack@eecs.umich.edu# modified or unmodified, in source code or in binary form. 123569Sgblack@eecs.umich.edu# 133569Sgblack@eecs.umich.edu# Copyright (c) 2015 The University of Bologna 143569Sgblack@eecs.umich.edu# All rights reserved. 153569Sgblack@eecs.umich.edu# 163569Sgblack@eecs.umich.edu# Redistribution and use in source and binary forms, with or without 173569Sgblack@eecs.umich.edu# modification, are permitted provided that the following conditions are 183569Sgblack@eecs.umich.edu# met: redistributions of source code must retain the above copyright 193569Sgblack@eecs.umich.edu# notice, this list of conditions and the following disclaimer; 203569Sgblack@eecs.umich.edu# redistributions in binary form must reproduce the above copyright 213569Sgblack@eecs.umich.edu# notice, this list of conditions and the following disclaimer in the 223569Sgblack@eecs.umich.edu# documentation and/or other materials provided with the distribution; 233569Sgblack@eecs.umich.edu# neither the name of the copyright holders nor the names of its 243569Sgblack@eecs.umich.edu# contributors may be used to endorse or promote products derived from 253569Sgblack@eecs.umich.edu# this software without specific prior written permission. 263569Sgblack@eecs.umich.edu# 273569Sgblack@eecs.umich.edu# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 283804Ssaidi@eecs.umich.edu# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 293569Sgblack@eecs.umich.edu# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 303569Sgblack@eecs.umich.edu# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 313918Ssaidi@eecs.umich.edu# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 323918Ssaidi@eecs.umich.edu# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 333804Ssaidi@eecs.umich.edu# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 343811Ssaidi@eecs.umich.edu# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 353569Sgblack@eecs.umich.edu# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 363824Ssaidi@eecs.umich.edu# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 373811Ssaidi@eecs.umich.edu# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 383811Ssaidi@eecs.umich.edu# 393823Ssaidi@eecs.umich.edu# Authors: Erfan Azarkhish 403823Ssaidi@eecs.umich.edu# Abdul Mutaal Ahmad 413823Ssaidi@eecs.umich.edu 424103Ssaidi@eecs.umich.edu# A Simplified model of a complete HMC device. Based on: 433569Sgblack@eecs.umich.edu# [1] http://www.hybridmemorycube.org/specification-download/ 443804Ssaidi@eecs.umich.edu# [2] High performance AXI-4.0 based interconnect for extensible smart memory 453804Ssaidi@eecs.umich.edu# cubes(E. Azarkhish et. al) 464088Sbinkertn@umich.edu# [3] Low-Power Hybrid Memory Cubes With Link Power Management and Two-Level 473569Sgblack@eecs.umich.edu# Prefetching (J. Ahn et. al) 485034Smilesck@eecs.umich.edu# [4] Memory-centric system interconnect design with Hybrid Memory Cubes 495358Sgblack@eecs.umich.edu# (G. Kim et. al) 503881Ssaidi@eecs.umich.edu# [5] Near Data Processing, Are we there yet? (M. Gokhale) 513804Ssaidi@eecs.umich.edu# http://www.cs.utah.edu/wondp/gokhale.pdf 523804Ssaidi@eecs.umich.edu# [6] openHMC - A Configurable Open-Source Hybrid Memory Cube Controller 533804Ssaidi@eecs.umich.edu# (J. Schmidt) 545555Snate@binkert.org# [7] Hybrid Memory Cube performance characterization on data-centric 553569Sgblack@eecs.umich.edu# workloads (M. Gokhale) 563804Ssaidi@eecs.umich.edu# 573918Ssaidi@eecs.umich.edu# This script builds a complete HMC device composed of vault controllers, 583881Ssaidi@eecs.umich.edu# serial links, the main internal crossbar, and an external hmc controller. 593881Ssaidi@eecs.umich.edu# 603881Ssaidi@eecs.umich.edu# - VAULT CONTROLLERS: 614990Sgblack@eecs.umich.edu# Instances of the HMC_2500_x32 class with their functionality specified in 624990Sgblack@eecs.umich.edu# dram_ctrl.cc 634990Sgblack@eecs.umich.edu# 644990Sgblack@eecs.umich.edu# - THE MAIN XBAR: 654990Sgblack@eecs.umich.edu# This component is simply an instance of the NoncoherentXBar class, and its 664990Sgblack@eecs.umich.edu# parameters are tuned to [2]. 674990Sgblack@eecs.umich.edu# 684990Sgblack@eecs.umich.edu# - SERIAL LINKS CONTROLLER: 694990Sgblack@eecs.umich.edu# SerialLink is a simple variation of the Bridge class, with the ability to 706022Sgblack@eecs.umich.edu# account for the latency of packet serialization and controller latency. We 716022Sgblack@eecs.umich.edu# assume that the serializer component at the transmitter side does not need 726022Sgblack@eecs.umich.edu# to receive the whole packet to start the serialization. But the 733804Ssaidi@eecs.umich.edu# deserializer waits for the complete packet to check its integrity first. 743569Sgblack@eecs.umich.edu# 753804Ssaidi@eecs.umich.edu# * Bandwidth of the serial links is not modeled in the SerialLink component 763804Ssaidi@eecs.umich.edu# itself. 773804Ssaidi@eecs.umich.edu# 783804Ssaidi@eecs.umich.edu# * Latency of serial link controller is composed of SerDes latency + link 793881Ssaidi@eecs.umich.edu# controller 803804Ssaidi@eecs.umich.edu# 813804Ssaidi@eecs.umich.edu# * It is inferred from the standard [1] and the literature [3] that serial 823804Ssaidi@eecs.umich.edu# links share the same address range and packets can travel over any of 833804Ssaidi@eecs.umich.edu# them so a load distribution mechanism is required among them. 843804Ssaidi@eecs.umich.edu# 853804Ssaidi@eecs.umich.edu# ----------------------------------------- 863804Ssaidi@eecs.umich.edu# | Host/HMC Controller | 873569Sgblack@eecs.umich.edu# | ---------------------- | 883569Sgblack@eecs.umich.edu# | | Link Aggregator | opt | 893804Ssaidi@eecs.umich.edu# | ---------------------- | 903804Ssaidi@eecs.umich.edu# | ---------------------- | 913826Ssaidi@eecs.umich.edu# | | Serial Link + Ser | * 4 | 923804Ssaidi@eecs.umich.edu# | ---------------------- | 933804Ssaidi@eecs.umich.edu# |--------------------------------------- 943826Ssaidi@eecs.umich.edu# ----------------------------------------- 953907Ssaidi@eecs.umich.edu# | Device 963826Ssaidi@eecs.umich.edu# | ---------------------- | 973811Ssaidi@eecs.umich.edu# | | Xbar | * 4 | 983836Ssaidi@eecs.umich.edu# | ---------------------- | 993915Ssaidi@eecs.umich.edu# | ---------------------- | 1003907Ssaidi@eecs.umich.edu# | | Vault Controller | * 16 | 1013881Ssaidi@eecs.umich.edu# | ---------------------- | 1023881Ssaidi@eecs.umich.edu# | ---------------------- | 1033881Ssaidi@eecs.umich.edu# | | Memory | | 1043881Ssaidi@eecs.umich.edu# | ---------------------- | 1053907Ssaidi@eecs.umich.edu# |---------------------------------------| 1063881Ssaidi@eecs.umich.edu# 1075555Snate@binkert.org# In this version we have present 3 different HMC archiecture along with 1085555Snate@binkert.org# alongwith their corresponding test script. 1095555Snate@binkert.org# 1103881Ssaidi@eecs.umich.edu# same: It has 4 crossbars in HMC memory. All the crossbars are connected 1113881Ssaidi@eecs.umich.edu# to each other, providing complete memory range. This archicture also covers 1123907Ssaidi@eecs.umich.edu# the added latency for sending a request to non-local vault(bridge in b/t 1133907Ssaidi@eecs.umich.edu# crossbars). All the 4 serial links can access complete memory. So each 1143907Ssaidi@eecs.umich.edu# link can be connected to separate processor. 1153907Ssaidi@eecs.umich.edu# 1163907Ssaidi@eecs.umich.edu# distributed: It has 4 crossbars inside the HMC. Crossbars are not 1173907Ssaidi@eecs.umich.edu# connected.Through each crossbar only local vaults can be accessed. But to 1183907Ssaidi@eecs.umich.edu# support this architecture we need a crossbar between serial links and 1193907Ssaidi@eecs.umich.edu# processor. 1203907Ssaidi@eecs.umich.edu# 1213907Ssaidi@eecs.umich.edu# mixed: This is a hybrid architecture. It has 4 crossbars inside the HMC. 1223907Ssaidi@eecs.umich.edu# 2 Crossbars are connected to only local vaults. From other 2 crossbar, a 1233907Ssaidi@eecs.umich.edu# request can be forwarded to any other vault. 1243907Ssaidi@eecs.umich.edu 1253907Ssaidi@eecs.umich.eduimport optparse 1263907Ssaidi@eecs.umich.edu 1273907Ssaidi@eecs.umich.eduimport m5 1283907Ssaidi@eecs.umich.edufrom m5.objects import * 1293907Ssaidi@eecs.umich.edu 1303907Ssaidi@eecs.umich.edu# A single Hybrid Memory Cube (HMC) 1313907Ssaidi@eecs.umich.educlass HMCSystem(SubSystem): 1323907Ssaidi@eecs.umich.edu #*****************************CROSSBAR PARAMETERS************************* 1333907Ssaidi@eecs.umich.edu # Flit size of the main interconnect [1] 1343881Ssaidi@eecs.umich.edu xbar_width = Param.Unsigned(32, "Data width of the main XBar (Bytes)") 1353881Ssaidi@eecs.umich.edu 1363881Ssaidi@eecs.umich.edu # Clock frequency of the main interconnect [1] 1373881Ssaidi@eecs.umich.edu # This crossbar, is placed on the logic-based of the HMC and it has its 1383881Ssaidi@eecs.umich.edu # own voltage and clock domains, different from the DRAM dies or from the 1393881Ssaidi@eecs.umich.edu # host. 1403881Ssaidi@eecs.umich.edu xbar_frequency = Param.Frequency('1GHz', "Clock Frequency of the main " 1413881Ssaidi@eecs.umich.edu "XBar") 1423881Ssaidi@eecs.umich.edu 1433881Ssaidi@eecs.umich.edu # Arbitration latency of the HMC XBar [1] 1443881Ssaidi@eecs.umich.edu xbar_frontend_latency = Param.Cycles(1, "Arbitration latency of the XBar") 1453881Ssaidi@eecs.umich.edu 1463907Ssaidi@eecs.umich.edu # Latency to forward a packet via the interconnect [1](two levels of FIFOs 1473811Ssaidi@eecs.umich.edu # at the input and output of the inteconnect) 1483826Ssaidi@eecs.umich.edu xbar_forward_latency = Param.Cycles(2, "Forward latency of the XBar") 1493826Ssaidi@eecs.umich.edu 1503826Ssaidi@eecs.umich.edu # Latency to forward a response via the interconnect [1](two levels of 1513826Ssaidi@eecs.umich.edu # FIFOs at the input and output of the inteconnect) 1523881Ssaidi@eecs.umich.edu xbar_response_latency = Param.Cycles(2, "Response latency of the XBar") 1533881Ssaidi@eecs.umich.edu 1543881Ssaidi@eecs.umich.edu # number of cross which connects 16 Vaults to serial link[7] 1553881Ssaidi@eecs.umich.edu number_mem_crossbar = Param.Unsigned(4, "Number of crossbar in HMC" 1563881Ssaidi@eecs.umich.edu ) 1573881Ssaidi@eecs.umich.edu 1583881Ssaidi@eecs.umich.edu #*****************************SERIAL LINK PARAMETERS*********************** 1593881Ssaidi@eecs.umich.edu # Number of serial links controllers [1] 1603881Ssaidi@eecs.umich.edu num_links_controllers = Param.Unsigned(4, "Number of serial links") 1613881Ssaidi@eecs.umich.edu 1623881Ssaidi@eecs.umich.edu # Number of packets (not flits) to store at the request side of the serial 1633881Ssaidi@eecs.umich.edu # link. This number should be adjusted to achive required bandwidth 1643881Ssaidi@eecs.umich.edu link_buffer_size_req = Param.Unsigned(10, "Number of packets to buffer " 1653881Ssaidi@eecs.umich.edu "at the request side of the serial link") 1663881Ssaidi@eecs.umich.edu 1673826Ssaidi@eecs.umich.edu # Number of packets (not flits) to store at the response side of the serial 1683826Ssaidi@eecs.umich.edu # link. This number should be adjusted to achive required bandwidth 1693826Ssaidi@eecs.umich.edu link_buffer_size_rsp = Param.Unsigned(10, "Number of packets to buffer " 1703826Ssaidi@eecs.umich.edu "at the response side of the serial link") 1713826Ssaidi@eecs.umich.edu 1723881Ssaidi@eecs.umich.edu # Latency of the serial link composed by SER/DES latency (1.6ns [4]) plus 1733569Sgblack@eecs.umich.edu # the PCB trace latency (3ns Estimated based on [5]) 1743569Sgblack@eecs.umich.edu link_latency = Param.Latency('4.6ns', "Latency of the serial links") 1753881Ssaidi@eecs.umich.edu 1763804Ssaidi@eecs.umich.edu # Clock frequency of the each serial link(SerDes) [1] 1773881Ssaidi@eecs.umich.edu link_frequency = Param.Frequency('10GHz', "Clock Frequency of the serial" 1783826Ssaidi@eecs.umich.edu "links") 1793881Ssaidi@eecs.umich.edu 1803881Ssaidi@eecs.umich.edu # Clock frequency of serial link Controller[6] 1813881Ssaidi@eecs.umich.edu # clk_hmc[Mhz]= num_lanes_per_link * lane_speed [Gbits/s] / 1823907Ssaidi@eecs.umich.edu # data_path_width * 10^6 1833907Ssaidi@eecs.umich.edu # clk_hmc[Mhz]= 16 * 10 Gbps / 256 * 10^6 = 625 Mhz 1843929Ssaidi@eecs.umich.edu link_controller_frequency = Param.Frequency('625MHz', 1853929Ssaidi@eecs.umich.edu "Clock Frequency of the link controller") 1863907Ssaidi@eecs.umich.edu 1873907Ssaidi@eecs.umich.edu # Latency of the serial link controller to process the packets[1][6] 1883804Ssaidi@eecs.umich.edu # (ClockDomain = 625 Mhz ) 1893804Ssaidi@eecs.umich.edu # used here for calculations only 1903881Ssaidi@eecs.umich.edu link_ctrl_latency = Param.Cycles(4, "The number of cycles required for the" 1913804Ssaidi@eecs.umich.edu "controller to process the packet") 1923804Ssaidi@eecs.umich.edu 1933804Ssaidi@eecs.umich.edu # total_ctrl_latency = link_ctrl_latency + link_latency 1943804Ssaidi@eecs.umich.edu # total_ctrl_latency = 4(Cycles) * 1.6 ns + 4.6 ns 1953804Ssaidi@eecs.umich.edu total_ctrl_latency = Param.Latency('11ns', "The latency experienced by" 1963804Ssaidi@eecs.umich.edu "every packet regardless of size of packet") 1973804Ssaidi@eecs.umich.edu 1983569Sgblack@eecs.umich.edu # Number of parallel lanes in each serial link [1] 1993863Ssaidi@eecs.umich.edu num_lanes_per_link = Param.Unsigned( 16, "Number of lanes per each link") 2003863Ssaidi@eecs.umich.edu 2013804Ssaidi@eecs.umich.edu # Number of serial links [1] 2025555Snate@binkert.org num_serial_links = Param.Unsigned(4, "Number of serial links") 2035555Snate@binkert.org 2043804Ssaidi@eecs.umich.edu # speed of each lane of serial link - SerDes serial interface 10 Gb/s 2053804Ssaidi@eecs.umich.edu serial_link_speed = Param.UInt64(10, "Gbs/s speed of each lane of" 2063804Ssaidi@eecs.umich.edu "serial link") 2073804Ssaidi@eecs.umich.edu 2083804Ssaidi@eecs.umich.edu #*****************************PERFORMANCE MONITORING************************ 2093569Sgblack@eecs.umich.edu # The main monitor behind the HMC Controller 2103804Ssaidi@eecs.umich.edu enable_global_monitor = Param.Bool(False, "The main monitor behind the " 2113804Ssaidi@eecs.umich.edu "HMC Controller") 2123804Ssaidi@eecs.umich.edu 2135555Snate@binkert.org # The link performance monitors 2145555Snate@binkert.org enable_link_monitor = Param.Bool(False, "The link monitors" ) 2153804Ssaidi@eecs.umich.edu 2163804Ssaidi@eecs.umich.edu # link aggregator enable - put a cross between buffers & links 2173804Ssaidi@eecs.umich.edu enable_link_aggr = Param.Bool(False, "The crossbar between port and " 2183804Ssaidi@eecs.umich.edu "Link Controller") 2193804Ssaidi@eecs.umich.edu 2203811Ssaidi@eecs.umich.edu enable_buff_div = Param.Bool(True, "Memory Range of Buffer is" 2213811Ssaidi@eecs.umich.edu "divided between total range") 2223804Ssaidi@eecs.umich.edu 2233804Ssaidi@eecs.umich.edu #*****************************HMC ARCHITECTURE ************************ 2245312Sgblack@eecs.umich.edu # Memory chunk for 16 vault - numbers of vault / number of crossbars 2253804Ssaidi@eecs.umich.edu mem_chunk = Param.Unsigned(4, "Chunk of memory range for each cross bar " 2263804Ssaidi@eecs.umich.edu "in arch 0") 2273804Ssaidi@eecs.umich.edu 2283804Ssaidi@eecs.umich.edu # size of req buffer within crossbar, used for modelling extra latency 2293804Ssaidi@eecs.umich.edu # when the reuqest go to non-local vault 2303804Ssaidi@eecs.umich.edu xbar_buffer_size_req = Param.Unsigned(10, "Number of packets to buffer " 2313804Ssaidi@eecs.umich.edu "at the request side of the crossbar") 2323811Ssaidi@eecs.umich.edu 2333804Ssaidi@eecs.umich.edu # size of response buffer within crossbar, used for modelling extra latency 2343804Ssaidi@eecs.umich.edu # when the response received from non-local vault 2353804Ssaidi@eecs.umich.edu xbar_buffer_size_resp = Param.Unsigned(10, "Number of packets to buffer " 2363804Ssaidi@eecs.umich.edu "at the response side of the crossbar") 2373804Ssaidi@eecs.umich.edu 2383826Ssaidi@eecs.umich.edu# configure host system with Serial Links 2393826Ssaidi@eecs.umich.edudef config_host_hmc(options, system): 2404070Ssaidi@eecs.umich.edu 2415555Snate@binkert.org system.hmc_host=HMCSystem() 2425555Snate@binkert.org 2434070Ssaidi@eecs.umich.edu try: 2443804Ssaidi@eecs.umich.edu system.hmc_host.enable_global_monitor = options.enable_global_monitor 2453804Ssaidi@eecs.umich.edu except: 2463804Ssaidi@eecs.umich.edu pass; 2473804Ssaidi@eecs.umich.edu 2483804Ssaidi@eecs.umich.edu try: 2493804Ssaidi@eecs.umich.edu system.hmc_host.enable_link_monitor = options.enable_link_monitor 2503804Ssaidi@eecs.umich.edu except: 2513804Ssaidi@eecs.umich.edu pass; 2523804Ssaidi@eecs.umich.edu 2533804Ssaidi@eecs.umich.edu # Serial link Controller with 16 SerDes links at 10 Gbps 2543804Ssaidi@eecs.umich.edu # with serial link ranges w.r.t to architecture 2553804Ssaidi@eecs.umich.edu system.hmc_host.seriallink = [SerialLink(ranges = options.ser_ranges[i], 2563826Ssaidi@eecs.umich.edu req_size=system.hmc_host.link_buffer_size_req, 2573826Ssaidi@eecs.umich.edu resp_size=system.hmc_host.link_buffer_size_rsp, 2583826Ssaidi@eecs.umich.edu num_lanes=system.hmc_host.num_lanes_per_link, 2593863Ssaidi@eecs.umich.edu link_speed=system.hmc_host.serial_link_speed, 2603826Ssaidi@eecs.umich.edu delay=system.hmc_host.total_ctrl_latency) 2613826Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.num_serial_links)] 2623826Ssaidi@eecs.umich.edu 2633826Ssaidi@eecs.umich.edu # enable global monitor 2643826Ssaidi@eecs.umich.edu if system.hmc_host.enable_global_monitor: 2653826Ssaidi@eecs.umich.edu system.hmc_host.lmonitor = [ CommMonitor() 2663826Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.num_serial_links)] 2673826Ssaidi@eecs.umich.edu 2683826Ssaidi@eecs.umich.edu # set the clock frequency for serial link 2693804Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.num_serial_links): 2703804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[i].clk_domain = SrcClockDomain(clock=system. 2713804Ssaidi@eecs.umich.edu hmc_host.link_controller_frequency, voltage_domain= 2723804Ssaidi@eecs.umich.edu VoltageDomain(voltage = '1V')) 2733804Ssaidi@eecs.umich.edu 2743804Ssaidi@eecs.umich.edu # Connect membus/traffic gen to Serial Link Controller for differrent HMC 2753804Ssaidi@eecs.umich.edu # architectures 2763863Ssaidi@eecs.umich.edu if options.arch == "distributed": 2773863Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.num_links_controllers): 2783863Ssaidi@eecs.umich.edu if system.hmc_host.enable_global_monitor: 2793836Ssaidi@eecs.umich.edu system.membus.master = system.hmc_host.lmonitor[i].slave 2803836Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[i].master = \ 2813804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[i].slave 2823804Ssaidi@eecs.umich.edu else: 2835312Sgblack@eecs.umich.edu system.membus.master = system.hmc_host.seriallink[i].slave 2843804Ssaidi@eecs.umich.edu if options.arch == "mixed": 2853804Ssaidi@eecs.umich.edu if system.hmc_host.enable_global_monitor: 2863804Ssaidi@eecs.umich.edu system.membus.master = system.hmc_host.lmonitor[0].slave 2873804Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[0].master = \ 2883804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[0].slave 2893804Ssaidi@eecs.umich.edu 2903804Ssaidi@eecs.umich.edu system.membus.master = system.hmc_host.lmonitor[1].slave 2913863Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[1].master = \ 2923804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[1].slave 2933804Ssaidi@eecs.umich.edu 2943804Ssaidi@eecs.umich.edu system.tgen[2].port = system.hmc_host.lmonitor[2].slave 2953804Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[2].master = \ 2963804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[2].slave 2973881Ssaidi@eecs.umich.edu 2983804Ssaidi@eecs.umich.edu system.tgen[3].port = system.hmc_host.lmonitor[3].slave 2993804Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[3].master = \ 3003804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[3].slave 3013804Ssaidi@eecs.umich.edu else: 3023804Ssaidi@eecs.umich.edu system.membus.master = system.hmc_host.seriallink[0].slave 3033804Ssaidi@eecs.umich.edu system.membus.master = system.hmc_host.seriallink[1].slave 3043804Ssaidi@eecs.umich.edu system.tgen[2].port = system.hmc_host.seriallink[2].slave 3053863Ssaidi@eecs.umich.edu system.tgen[3].port = system.hmc_host.seriallink[3].slave 3063863Ssaidi@eecs.umich.edu if options.arch == "same" : 3073836Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.num_links_controllers): 3085555Snate@binkert.org if system.hmc_host.enable_global_monitor: 3093804Ssaidi@eecs.umich.edu system.tgen[i].port = system.hmc_host.lmonitor[i].slave 3103804Ssaidi@eecs.umich.edu system.hmc_host.lmonitor[i].master = \ 3113881Ssaidi@eecs.umich.edu system.hmc_host.seriallink[i].slave 3123881Ssaidi@eecs.umich.edu else: 3133881Ssaidi@eecs.umich.edu system.tgen[i].port = system.hmc_host.seriallink[i].slave 3143804Ssaidi@eecs.umich.edu 3153804Ssaidi@eecs.umich.edu return system 3163804Ssaidi@eecs.umich.edu 3173804Ssaidi@eecs.umich.edu# Create an HMC device and attach it to the current system 3183804Ssaidi@eecs.umich.edudef config_hmc(options, system, hmc_host): 3193804Ssaidi@eecs.umich.edu 3203804Ssaidi@eecs.umich.edu # Create HMC device 3213804Ssaidi@eecs.umich.edu system.hmc_dev = HMCSystem() 3223804Ssaidi@eecs.umich.edu 3233804Ssaidi@eecs.umich.edu # Global monitor 3243804Ssaidi@eecs.umich.edu try: 3253804Ssaidi@eecs.umich.edu system.hmc_dev.enable_global_monitor = options.enable_global_monitor 3263804Ssaidi@eecs.umich.edu except: 3273863Ssaidi@eecs.umich.edu pass; 3283836Ssaidi@eecs.umich.edu 3295555Snate@binkert.org try: 3305288Sgblack@eecs.umich.edu system.hmc_dev.enable_link_monitor = options.enable_link_monitor 3315288Sgblack@eecs.umich.edu except: 3325288Sgblack@eecs.umich.edu pass; 3333804Ssaidi@eecs.umich.edu 3343804Ssaidi@eecs.umich.edu 3353804Ssaidi@eecs.umich.edu if system.hmc_dev.enable_link_monitor: 3363804Ssaidi@eecs.umich.edu system.hmc_dev.lmonitor = [ CommMonitor() 3373804Ssaidi@eecs.umich.edu for i in xrange(system.hmc_dev.num_links_controllers)] 3383804Ssaidi@eecs.umich.edu 3393804Ssaidi@eecs.umich.edu # 4 HMC Crossbars located in its logic-base (LoB) 3403804Ssaidi@eecs.umich.edu system.hmc_dev.xbar = [ NoncoherentXBar(width=system.hmc_dev.xbar_width, 3413804Ssaidi@eecs.umich.edu frontend_latency=system.hmc_dev.xbar_frontend_latency, 3423804Ssaidi@eecs.umich.edu forward_latency=system.hmc_dev.xbar_forward_latency, 3433804Ssaidi@eecs.umich.edu response_latency=system.hmc_dev.xbar_response_latency ) 3443804Ssaidi@eecs.umich.edu for i in xrange(system.hmc_host.number_mem_crossbar)] 3453804Ssaidi@eecs.umich.edu 3463836Ssaidi@eecs.umich.edu for i in xrange(system.hmc_dev.number_mem_crossbar): 3475555Snate@binkert.org system.hmc_dev.xbar[i].clk_domain = SrcClockDomain( 3483836Ssaidi@eecs.umich.edu clock=system.hmc_dev.xbar_frequency,voltage_domain= 3495555Snate@binkert.org VoltageDomain(voltage='1V')) 3503881Ssaidi@eecs.umich.edu 3513881Ssaidi@eecs.umich.edu # Attach 4 serial link to 4 crossbar/s 3523804Ssaidi@eecs.umich.edu for i in xrange(system.hmc_dev.num_serial_links): 3533907Ssaidi@eecs.umich.edu if system.hmc_dev.enable_link_monitor: 3543804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[i].master = \ 3553804Ssaidi@eecs.umich.edu system.hmc_dev.lmonitor[i].slave 3563804Ssaidi@eecs.umich.edu system.hmc_dev.lmonitor[i].master = system.hmc_dev.xbar[i].slave 3573804Ssaidi@eecs.umich.edu else: 3583804Ssaidi@eecs.umich.edu system.hmc_host.seriallink[i].master = system.hmc_dev.xbar[i].slave 3595555Snate@binkert.org 3605555Snate@binkert.org # Connecting xbar with each other for request arriving at the wrong xbar, 3613881Ssaidi@eecs.umich.edu # then it will be forward to correct xbar. Bridge is used to connect xbars 3623881Ssaidi@eecs.umich.edu if options.arch == "same": 3633881Ssaidi@eecs.umich.edu numx = len(system.hmc_dev.xbar) 3643804Ssaidi@eecs.umich.edu 3653881Ssaidi@eecs.umich.edu # create a list of buffers 3663881Ssaidi@eecs.umich.edu system.hmc_dev.buffers = [ Bridge( 3673881Ssaidi@eecs.umich.edu req_size=system.hmc_dev.xbar_buffer_size_req, 3683881Ssaidi@eecs.umich.edu resp_size=system.hmc_dev.xbar_buffer_size_resp) 3693804Ssaidi@eecs.umich.edu for i in xrange(numx * (system.hmc_dev.mem_chunk - 1))] 3703804Ssaidi@eecs.umich.edu 3713804Ssaidi@eecs.umich.edu # Buffer iterator 3725555Snate@binkert.org it = iter(range(len(system.hmc_dev.buffers))) 3735555Snate@binkert.org 3743804Ssaidi@eecs.umich.edu # necesarry to add system_port to one of the xbar 3753804Ssaidi@eecs.umich.edu system.system_port = system.hmc_dev.xbar[3].slave 3763881Ssaidi@eecs.umich.edu 3773881Ssaidi@eecs.umich.edu # iterate over all the crossbars and connect them as required 3783804Ssaidi@eecs.umich.edu for i in range(numx): 3793881Ssaidi@eecs.umich.edu for j in range(numx): 3803881Ssaidi@eecs.umich.edu # connect xbar to all other xbars except itself 3813881Ssaidi@eecs.umich.edu if i != j: 3823804Ssaidi@eecs.umich.edu # get the next index of buffer 3833804Ssaidi@eecs.umich.edu index = it.next() 3843804Ssaidi@eecs.umich.edu 3853804Ssaidi@eecs.umich.edu # Change the default values for ranges of bridge 3863804Ssaidi@eecs.umich.edu system.hmc_dev.buffers[index].ranges = system.mem_ranges[ 3873804Ssaidi@eecs.umich.edu j * int(system.hmc_dev.mem_chunk): 3883804Ssaidi@eecs.umich.edu (j + 1) * int(system.hmc_dev.mem_chunk)] 3893804Ssaidi@eecs.umich.edu 3903804Ssaidi@eecs.umich.edu # Connect the bridge between corssbars 3913804Ssaidi@eecs.umich.edu system.hmc_dev.xbar[i].master = system.hmc_dev.buffers[ 3923804Ssaidi@eecs.umich.edu index].slave 3933804Ssaidi@eecs.umich.edu system.hmc_dev.buffers[ 3943804Ssaidi@eecs.umich.edu index].master = system.hmc_dev.xbar[j].slave 3953804Ssaidi@eecs.umich.edu else: 3963804Ssaidi@eecs.umich.edu # Don't connect the xbar to itself 3973804Ssaidi@eecs.umich.edu pass 3984990Sgblack@eecs.umich.edu 3993804Ssaidi@eecs.umich.edu # Two crossbars are connected to all other crossbars-Other 2 vault 4003804Ssaidi@eecs.umich.edu # can only direct traffic to it local vaults 4013804Ssaidi@eecs.umich.edu if options.arch == "mixed": 4023804Ssaidi@eecs.umich.edu 4033804Ssaidi@eecs.umich.edu system.hmc_dev.buffer30 = Bridge(ranges=system.mem_ranges[0:4]) 4043804Ssaidi@eecs.umich.edu system.hmc_dev.xbar[3].master = system.hmc_dev.buffer30.slave 4053804Ssaidi@eecs.umich.edu system.hmc_dev.buffer30.master = system.hmc_dev.xbar[0].slave 4063804Ssaidi@eecs.umich.edu 4073804Ssaidi@eecs.umich.edu system.hmc_dev.buffer31 = Bridge(ranges=system.mem_ranges[4:8]) 4083804Ssaidi@eecs.umich.edu system.hmc_dev.xbar[3].master = system.hmc_dev.buffer31.slave 4093804Ssaidi@eecs.umich.edu system.hmc_dev.buffer31.master = system.hmc_dev.xbar[1].slave 4103804Ssaidi@eecs.umich.edu 4113804Ssaidi@eecs.umich.edu system.hmc_dev.buffer32 = Bridge(ranges=system.mem_ranges[8:12]) 4123804Ssaidi@eecs.umich.edu system.hmc_dev.xbar[3].master = system.hmc_dev.buffer32.slave 4133804Ssaidi@eecs.umich.edu system.hmc_dev.buffer32.master = system.hmc_dev.xbar[2].slave 4143826Ssaidi@eecs.umich.edu 4154990Sgblack@eecs.umich.edu 4163826Ssaidi@eecs.umich.edu system.hmc_dev.buffer20 = Bridge(ranges=system.mem_ranges[0:4]) 4173916Ssaidi@eecs.umich.edu system.hmc_dev.xbar[2].master = system.hmc_dev.buffer20.slave 4183916Ssaidi@eecs.umich.edu system.hmc_dev.buffer20.master = system.hmc_dev.xbar[0].slave 4193916Ssaidi@eecs.umich.edu 4204990Sgblack@eecs.umich.edu system.hmc_dev.buffer21 = Bridge(ranges=system.mem_ranges[4:8]) 4213826Ssaidi@eecs.umich.edu system.hmc_dev.xbar[2].master = system.hmc_dev.buffer21.slave 4223804Ssaidi@eecs.umich.edu system.hmc_dev.buffer21.master = system.hmc_dev.xbar[1].slave 4233804Ssaidi@eecs.umich.edu 4246022Sgblack@eecs.umich.edu system.hmc_dev.buffer23 = Bridge(ranges=system.mem_ranges[12:16]) 4253804Ssaidi@eecs.umich.edu system.hmc_dev.xbar[2].master = system.hmc_dev.buffer23.slave 4263804Ssaidi@eecs.umich.edu system.hmc_dev.buffer23.master = system.hmc_dev.xbar[3].slave 4276022Sgblack@eecs.umich.edu 4283811Ssaidi@eecs.umich.edu