HMC.py revision 11292
111183Serfan.azarkhish@unibo.it# Copyright (c) 2012-2013 ARM Limited 211183Serfan.azarkhish@unibo.it# All rights reserved. 311183Serfan.azarkhish@unibo.it# 411183Serfan.azarkhish@unibo.it# The license below extends only to copyright in the software and shall 511183Serfan.azarkhish@unibo.it# not be construed as granting a license to any other intellectual 611183Serfan.azarkhish@unibo.it# property including but not limited to intellectual property relating 711183Serfan.azarkhish@unibo.it# to a hardware implementation of the functionality of the software 811183Serfan.azarkhish@unibo.it# licensed hereunder. You may use the software subject to the license 911183Serfan.azarkhish@unibo.it# terms below provided that you ensure that this notice is replicated 1011183Serfan.azarkhish@unibo.it# unmodified and in its entirety in all distributions of the software, 1111183Serfan.azarkhish@unibo.it# modified or unmodified, in source code or in binary form. 1211183Serfan.azarkhish@unibo.it# 1311183Serfan.azarkhish@unibo.it# Copyright (c) 2015 The University of Bologna 1411183Serfan.azarkhish@unibo.it# All rights reserved. 1511183Serfan.azarkhish@unibo.it# 1611183Serfan.azarkhish@unibo.it# Redistribution and use in source and binary forms, with or without 1711183Serfan.azarkhish@unibo.it# modification, are permitted provided that the following conditions are 1811183Serfan.azarkhish@unibo.it# met: redistributions of source code must retain the above copyright 1911183Serfan.azarkhish@unibo.it# notice, this list of conditions and the following disclaimer; 2011183Serfan.azarkhish@unibo.it# redistributions in binary form must reproduce the above copyright 2111183Serfan.azarkhish@unibo.it# notice, this list of conditions and the following disclaimer in the 2211183Serfan.azarkhish@unibo.it# documentation and/or other materials provided with the distribution; 2311183Serfan.azarkhish@unibo.it# neither the name of the copyright holders nor the names of its 2411183Serfan.azarkhish@unibo.it# contributors may be used to endorse or promote products derived from 2511183Serfan.azarkhish@unibo.it# this software without specific prior written permission. 2611183Serfan.azarkhish@unibo.it# 2711183Serfan.azarkhish@unibo.it# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 2811183Serfan.azarkhish@unibo.it# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 2911183Serfan.azarkhish@unibo.it# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 3011183Serfan.azarkhish@unibo.it# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 3111183Serfan.azarkhish@unibo.it# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 3211183Serfan.azarkhish@unibo.it# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 3311183Serfan.azarkhish@unibo.it# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 3411183Serfan.azarkhish@unibo.it# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 3511183Serfan.azarkhish@unibo.it# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 3611183Serfan.azarkhish@unibo.it# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 3711183Serfan.azarkhish@unibo.it# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 3811183Serfan.azarkhish@unibo.it# 3911183Serfan.azarkhish@unibo.it# Authors: Erfan Azarkhish 4011183Serfan.azarkhish@unibo.it 4111183Serfan.azarkhish@unibo.it# A Simplified model of a complete HMC device. Based on: 4211183Serfan.azarkhish@unibo.it# [1] http://www.hybridmemorycube.org/specification-download/ 4311183Serfan.azarkhish@unibo.it# [2] High performance AXI-4.0 based interconnect for extensible smart memory 4411183Serfan.azarkhish@unibo.it# cubes(E. Azarkhish et. al) 4511183Serfan.azarkhish@unibo.it# [3] Low-Power Hybrid Memory Cubes With Link Power Management and Two-Level 4611183Serfan.azarkhish@unibo.it# Prefetching (J. Ahn et. al) 4711183Serfan.azarkhish@unibo.it# [4] Memory-centric system interconnect design with Hybrid Memory Cubes 4811183Serfan.azarkhish@unibo.it# (G. Kim et. al) 4911183Serfan.azarkhish@unibo.it# [5] Near Data Processing, Are we there yet? (M. Gokhale) 5011183Serfan.azarkhish@unibo.it# http://www.cs.utah.edu/wondp/gokhale.pdf 5111183Serfan.azarkhish@unibo.it# 5211183Serfan.azarkhish@unibo.it# This script builds a complete HMC device composed of vault controllers, 5311183Serfan.azarkhish@unibo.it# serial links, the main internal crossbar, and an external hmc controller. 5411183Serfan.azarkhish@unibo.it# 5511183Serfan.azarkhish@unibo.it# - VAULT CONTROLLERS: 5611183Serfan.azarkhish@unibo.it# Instances of the HMC_2500_x32 class with their functionality specified in 5711183Serfan.azarkhish@unibo.it# dram_ctrl.cc 5811183Serfan.azarkhish@unibo.it# 5911183Serfan.azarkhish@unibo.it# - THE MAIN XBAR: 6011183Serfan.azarkhish@unibo.it# This component is simply an instance of the NoncoherentXBar class, and its 6111183Serfan.azarkhish@unibo.it# parameters are tuned to [2]. 6211183Serfan.azarkhish@unibo.it# 6311183Serfan.azarkhish@unibo.it# - SERIAL LINKS: 6411183Serfan.azarkhish@unibo.it# SerialLink is a simple variation of the Bridge class, with the ability to 6511183Serfan.azarkhish@unibo.it# account for the latency of packet serialization. We assume that the 6611183Serfan.azarkhish@unibo.it# serializer component at the transmitter side does not need to receive the 6711183Serfan.azarkhish@unibo.it# whole packet to start the serialization. But the deserializer waits for 6811183Serfan.azarkhish@unibo.it# the complete packet to check its integrity first. 6911183Serfan.azarkhish@unibo.it# * Bandwidth of the serial links is not modeled in the SerialLink component 7011183Serfan.azarkhish@unibo.it# itself. Instead bandwidth/port of the HMCController has been adjusted to 7111183Serfan.azarkhish@unibo.it# reflect the bandwidth delivered by 1 serial link. 7211183Serfan.azarkhish@unibo.it# 7311183Serfan.azarkhish@unibo.it# - HMC CONTROLLER: 7411183Serfan.azarkhish@unibo.it# Contains a large buffer (modeled with Bridge) to hide the access latency 7511183Serfan.azarkhish@unibo.it# of the memory cube. Plus it simply forwards the packets to the serial 7611183Serfan.azarkhish@unibo.it# links in a round-robin fashion to balance load among them. 7711183Serfan.azarkhish@unibo.it# * It is inferred from the standard [1] and the literature [3] that serial 7811183Serfan.azarkhish@unibo.it# links share the same address range and packets can travel over any of 7911183Serfan.azarkhish@unibo.it# them so a load distribution mechanism is required among them. 8011183Serfan.azarkhish@unibo.it 8111183Serfan.azarkhish@unibo.itimport optparse 8211183Serfan.azarkhish@unibo.it 8311183Serfan.azarkhish@unibo.itimport m5 8411183Serfan.azarkhish@unibo.itfrom m5.objects import * 8511183Serfan.azarkhish@unibo.it 8611183Serfan.azarkhish@unibo.it# A single Hybrid Memory Cube (HMC) 8711292Sandreas.hansson@arm.comclass HMCSystem(SubSystem): 8811183Serfan.azarkhish@unibo.it #*****************************CROSSBAR PARAMETERS************************* 8911183Serfan.azarkhish@unibo.it # Flit size of the main interconnect [1] 9011292Sandreas.hansson@arm.com xbar_width = Param.Unsigned(32, "Data width of the main XBar (Bytes)") 9111183Serfan.azarkhish@unibo.it 9211183Serfan.azarkhish@unibo.it # Clock frequency of the main interconnect [1] 9311183Serfan.azarkhish@unibo.it # This crossbar, is placed on the logic-based of the HMC and it has its 9411183Serfan.azarkhish@unibo.it # own voltage and clock domains, different from the DRAM dies or from the 9511183Serfan.azarkhish@unibo.it # host. 9611183Serfan.azarkhish@unibo.it xbar_frequency = Param.Frequency('1GHz', "Clock Frequency of the main " 9711183Serfan.azarkhish@unibo.it "XBar") 9811183Serfan.azarkhish@unibo.it 9911183Serfan.azarkhish@unibo.it # Arbitration latency of the HMC XBar [1] 10011183Serfan.azarkhish@unibo.it xbar_frontend_latency = Param.Cycles(1, "Arbitration latency of the XBar") 10111183Serfan.azarkhish@unibo.it 10211183Serfan.azarkhish@unibo.it # Latency to forward a packet via the interconnect [1](two levels of FIFOs 10311183Serfan.azarkhish@unibo.it # at the input and output of the inteconnect) 10411183Serfan.azarkhish@unibo.it xbar_forward_latency = Param.Cycles(2, "Forward latency of the XBar") 10511183Serfan.azarkhish@unibo.it 10611183Serfan.azarkhish@unibo.it # Latency to forward a response via the interconnect [1](two levels of 10711183Serfan.azarkhish@unibo.it # FIFOs at the input and output of the inteconnect) 10811183Serfan.azarkhish@unibo.it xbar_response_latency = Param.Cycles(2, "Response latency of the XBar") 10911183Serfan.azarkhish@unibo.it 11011183Serfan.azarkhish@unibo.it #*****************************SERIAL LINK PARAMETERS********************** 11111183Serfan.azarkhish@unibo.it # Number of serial links [1] 11211183Serfan.azarkhish@unibo.it num_serial_links = Param.Unsigned(4, "Number of serial links") 11311183Serfan.azarkhish@unibo.it 11411183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the request side of the serial 11511183Serfan.azarkhish@unibo.it # link. This number should be adjusted to achive required bandwidth 11611292Sandreas.hansson@arm.com link_buffer_size_req = Param.Unsigned(16, "Number of packets to buffer " 11711183Serfan.azarkhish@unibo.it "at the request side of the serial link") 11811183Serfan.azarkhish@unibo.it 11911183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the response side of the serial 12011183Serfan.azarkhish@unibo.it # link. This number should be adjusted to achive required bandwidth 12111292Sandreas.hansson@arm.com link_buffer_size_rsp = Param.Unsigned(16, "Number of packets to buffer " 12211183Serfan.azarkhish@unibo.it "at the response side of the serial link") 12311183Serfan.azarkhish@unibo.it 12411183Serfan.azarkhish@unibo.it # Latency of the serial link composed by SER/DES latency (1.6ns [4]) plus 12511183Serfan.azarkhish@unibo.it # the PCB trace latency (3ns Estimated based on [5]) 12611183Serfan.azarkhish@unibo.it link_latency = Param.Latency('4.6ns', "Latency of the serial links") 12711183Serfan.azarkhish@unibo.it 12811183Serfan.azarkhish@unibo.it # Header overhead of the serial links: Header size is 128bits in HMC [1], 12911183Serfan.azarkhish@unibo.it # and we have 16 lanes, so the overhead is 8 cycles 13011183Serfan.azarkhish@unibo.it link_overhead = Param.Cycles(8, "The number of cycles required to" 13111183Serfan.azarkhish@unibo.it " transmit the packet header over the serial link") 13211183Serfan.azarkhish@unibo.it 13311183Serfan.azarkhish@unibo.it # Clock frequency of the serial links [1] 13411183Serfan.azarkhish@unibo.it link_frequency = Param.Frequency('10GHz', "Clock Frequency of the serial" 13511183Serfan.azarkhish@unibo.it "links") 13611183Serfan.azarkhish@unibo.it 13711183Serfan.azarkhish@unibo.it # Number of parallel lanes in each serial link [1] 13811292Sandreas.hansson@arm.com num_lanes_per_link = Param.Unsigned(16, "Number of lanes per each link") 13911183Serfan.azarkhish@unibo.it 14011183Serfan.azarkhish@unibo.it # Number of serial links [1] 14111292Sandreas.hansson@arm.com num_serial_links = Param.Unsigned(4, "Number of serial links") 14211183Serfan.azarkhish@unibo.it 14311183Serfan.azarkhish@unibo.it #*****************************HMC CONTROLLER PARAMETERS******************* 14411183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the HMC controller. This 14511183Serfan.azarkhish@unibo.it # number should be high enough to be able to hide the high latency of HMC 14611292Sandreas.hansson@arm.com ctrl_buffer_size_req = Param.Unsigned(256, "Number of packets to buffer " 14711183Serfan.azarkhish@unibo.it "at the HMC controller (request side)") 14811183Serfan.azarkhish@unibo.it 14911183Serfan.azarkhish@unibo.it # Number of packets (not flits) to store at the response side of the HMC 15011183Serfan.azarkhish@unibo.it # controller. 15111292Sandreas.hansson@arm.com ctrl_buffer_size_rsp = Param.Unsigned(256, "Number of packets to buffer " 15211183Serfan.azarkhish@unibo.it "at the HMC controller (response side)") 15311183Serfan.azarkhish@unibo.it 15411183Serfan.azarkhish@unibo.it # Latency of the HMC controller to process the packets 15511183Serfan.azarkhish@unibo.it # (ClockDomain = Host clock domain) 15611183Serfan.azarkhish@unibo.it ctrl_latency = Param.Cycles(4, "The number of cycles required for the " 15711183Serfan.azarkhish@unibo.it " controller to process the packet") 15811183Serfan.azarkhish@unibo.it 15911183Serfan.azarkhish@unibo.it # Wiring latency from the SoC crossbar to the HMC controller 16011183Serfan.azarkhish@unibo.it ctrl_static_latency = Param.Latency('500ps', "Static latency of the HMC" 16111183Serfan.azarkhish@unibo.it "controller") 16211183Serfan.azarkhish@unibo.it 16311183Serfan.azarkhish@unibo.it #*****************************PERFORMANCE MONITORING********************** 16411183Serfan.azarkhish@unibo.it # The main monitor behind the HMC Controller 16511183Serfan.azarkhish@unibo.it enable_global_monitor = Param.Bool(True, "The main monitor behind the " 16611183Serfan.azarkhish@unibo.it "HMC Controller") 16711183Serfan.azarkhish@unibo.it 16811183Serfan.azarkhish@unibo.it # The link performance monitors 16911292Sandreas.hansson@arm.com enable_link_monitor = Param.Bool(True, "The link monitors") 17011183Serfan.azarkhish@unibo.it 17111183Serfan.azarkhish@unibo.it# Create an HMC device and attach it to the current system 17211183Serfan.azarkhish@unibo.itdef config_hmc(options, system): 17311183Serfan.azarkhish@unibo.it 17411292Sandreas.hansson@arm.com system.hmc = HMCSystem() 17511183Serfan.azarkhish@unibo.it 17611183Serfan.azarkhish@unibo.it system.buffer = Bridge(ranges=system.mem_ranges, 17711292Sandreas.hansson@arm.com req_size=system.hmc.ctrl_buffer_size_req, 17811292Sandreas.hansson@arm.com resp_size=system.hmc.ctrl_buffer_size_rsp, 17911292Sandreas.hansson@arm.com delay=system.hmc.ctrl_static_latency) 18011183Serfan.azarkhish@unibo.it try: 18111183Serfan.azarkhish@unibo.it system.hmc.enable_global_monitor = options.enable_global_monitor 18211183Serfan.azarkhish@unibo.it except: 18311183Serfan.azarkhish@unibo.it pass; 18411183Serfan.azarkhish@unibo.it 18511183Serfan.azarkhish@unibo.it try: 18611183Serfan.azarkhish@unibo.it system.hmc.enable_link_monitor = options.enable_link_monitor 18711183Serfan.azarkhish@unibo.it except: 18811183Serfan.azarkhish@unibo.it pass; 18911183Serfan.azarkhish@unibo.it 19011183Serfan.azarkhish@unibo.it system.membus.master = system.buffer.slave 19111183Serfan.azarkhish@unibo.it 19211183Serfan.azarkhish@unibo.it # The HMC controller (Clock domain is the same as the host) 19311183Serfan.azarkhish@unibo.it system.hmccontroller = HMCController(width=(system.hmc.num_lanes_per_link. 19411183Serfan.azarkhish@unibo.it value * system.hmc.num_serial_links/8), 19511183Serfan.azarkhish@unibo.it frontend_latency=system.hmc.ctrl_latency, 19611183Serfan.azarkhish@unibo.it forward_latency=system.hmc.link_overhead, 19711183Serfan.azarkhish@unibo.it response_latency=system.hmc.link_overhead) 19811183Serfan.azarkhish@unibo.it 19911183Serfan.azarkhish@unibo.it system.hmccontroller.clk_domain = SrcClockDomain(clock=system.hmc. 20011183Serfan.azarkhish@unibo.it link_frequency, voltage_domain = VoltageDomain(voltage = '1V')) 20111183Serfan.azarkhish@unibo.it 20211183Serfan.azarkhish@unibo.it # Serial Links 20311183Serfan.azarkhish@unibo.it system.hmc.seriallink =[ SerialLink(ranges = system.mem_ranges, 20411183Serfan.azarkhish@unibo.it req_size=system.hmc.link_buffer_size_req, 20511183Serfan.azarkhish@unibo.it resp_size=system.hmc.link_buffer_size_rsp, 20611183Serfan.azarkhish@unibo.it num_lanes=system.hmc.num_lanes_per_link, 20711183Serfan.azarkhish@unibo.it delay=system.hmc.link_latency) 20811183Serfan.azarkhish@unibo.it for i in xrange(system.hmc.num_serial_links)] 20911183Serfan.azarkhish@unibo.it 21011183Serfan.azarkhish@unibo.it if system.hmc.enable_link_monitor: 21111183Serfan.azarkhish@unibo.it system.hmc.lmonitor = [ CommMonitor() 21211183Serfan.azarkhish@unibo.it for i in xrange(system.hmc.num_serial_links)] 21311183Serfan.azarkhish@unibo.it 21411183Serfan.azarkhish@unibo.it # The HMC Crossbar located in its logic-base (LoB) 21511183Serfan.azarkhish@unibo.it system.hmc.xbar = NoncoherentXBar(width = system.hmc.xbar_width, 21611183Serfan.azarkhish@unibo.it frontend_latency=system.hmc.xbar_frontend_latency, 21711183Serfan.azarkhish@unibo.it forward_latency=system.hmc.xbar_forward_latency, 21811183Serfan.azarkhish@unibo.it response_latency=system.hmc.xbar_response_latency ) 21911183Serfan.azarkhish@unibo.it system.hmc.xbar.clk_domain = SrcClockDomain(clock = 22011183Serfan.azarkhish@unibo.it system.hmc.xbar_frequency, voltage_domain = 22111183Serfan.azarkhish@unibo.it VoltageDomain(voltage = '1V')) 22211183Serfan.azarkhish@unibo.it 22311183Serfan.azarkhish@unibo.it if system.hmc.enable_global_monitor: 22411183Serfan.azarkhish@unibo.it system.gmonitor = CommMonitor() 22511183Serfan.azarkhish@unibo.it system.buffer.master = system.gmonitor.slave 22611183Serfan.azarkhish@unibo.it system.gmonitor.master = system.hmccontroller.slave 22711183Serfan.azarkhish@unibo.it else: 22811183Serfan.azarkhish@unibo.it system.hmccontroller.slave = system.buffer.master 22911183Serfan.azarkhish@unibo.it 23011183Serfan.azarkhish@unibo.it for i in xrange(system.hmc.num_serial_links): 23111183Serfan.azarkhish@unibo.it system.hmccontroller.master = system.hmc.seriallink[i].slave 23211183Serfan.azarkhish@unibo.it system.hmc.seriallink[i].clk_domain = system.hmccontroller.clk_domain; 23311183Serfan.azarkhish@unibo.it if system.hmc.enable_link_monitor: 23411183Serfan.azarkhish@unibo.it system.hmc.seriallink[i].master = system.hmc.lmonitor[i].slave 23511183Serfan.azarkhish@unibo.it system.hmc.lmonitor[i].master = system.hmc.xbar.slave 23611183Serfan.azarkhish@unibo.it else: 23711183Serfan.azarkhish@unibo.it system.hmc.seriallink[i].master = system.hmc.xbar.slave 238