dist_iface.hh revision 10923
12315SN/A/* 28733Sgeoffrey.blake@arm.com * Copyright (c) 2015 ARM Limited 39913Ssteve.reinhardt@amd.com * All rights reserved 48733Sgeoffrey.blake@arm.com * 58733Sgeoffrey.blake@arm.com * The license below extends only to copyright in the software and shall 68733Sgeoffrey.blake@arm.com * not be construed as granting a license to any other intellectual 78733Sgeoffrey.blake@arm.com * property including but not limited to intellectual property relating 88733Sgeoffrey.blake@arm.com * to a hardware implementation of the functionality of the software 98733Sgeoffrey.blake@arm.com * licensed hereunder. You may use the software subject to the license 108733Sgeoffrey.blake@arm.com * terms below provided that you ensure that this notice is replicated 118733Sgeoffrey.blake@arm.com * unmodified and in its entirety in all distributions of the software, 128733Sgeoffrey.blake@arm.com * modified or unmodified, in source code or in binary form. 138733Sgeoffrey.blake@arm.com * 148733Sgeoffrey.blake@arm.com * Redistribution and use in source and binary forms, with or without 152332SN/A * modification, are permitted provided that the following conditions are 162315SN/A * met: redistributions of source code must retain the above copyright 172315SN/A * notice, this list of conditions and the following disclaimer; 182315SN/A * redistributions in binary form must reproduce the above copyright 192315SN/A * notice, this list of conditions and the following disclaimer in the 202315SN/A * documentation and/or other materials provided with the distribution; 212315SN/A * neither the name of the copyright holders nor the names of its 222315SN/A * contributors may be used to endorse or promote products derived from 232315SN/A * this software without specific prior written permission. 242315SN/A * 252315SN/A * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 262315SN/A * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 272315SN/A * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 282315SN/A * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 292315SN/A * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 302315SN/A * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 312315SN/A * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 322315SN/A * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 332315SN/A * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 342315SN/A * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 352315SN/A * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 362315SN/A * 372315SN/A * Authors: Gabor Dozsa 382315SN/A */ 392315SN/A 402689SN/A/* @file 412689SN/A * The interface class for multi gem5 simulations. 428733Sgeoffrey.blake@arm.com * 432315SN/A * Multi gem5 is an extension to gem5 to enable parallel simulation of a 442315SN/A * distributed system (e.g. simulation of a pool of machines 459944Smatt.horsnell@ARM.com * connected by Ethernet links). A multi gem5 run consists of seperate gem5 469944Smatt.horsnell@ARM.com * processes running in parallel. Each gem5 process executes 479944Smatt.horsnell@ARM.com * the simulation of a component of the simulated distributed system. 482315SN/A * (An example component can be a multi-core board with an Ethernet NIC.) 492315SN/A * The MultiIface class below provides services to transfer data and 502315SN/A * control messages among the gem5 processes. The main such services are 518888Sgeoffrey.blake@arm.com * as follows. 528793Sgblack@eecs.umich.edu * 532315SN/A * 1. Send a data packet coming from a simulated Ethernet link. The packet 546658Snate@binkert.org * will be transferred to (all) the target(s) gem5 processes. The send 552315SN/A * operation is always performed by the simulation thread, i.e. the gem5 568733Sgeoffrey.blake@arm.com * thread that is processing the event queue associated with the simulated 579913Ssteve.reinhardt@amd.com * Ethernet link. 582683SN/A * 598229Snate@binkert.org * 2. Spawn a receiver thread to process messages coming in from the 602680SN/A * from other gem5 processes. Each simulated Ethernet link has its own 618733Sgeoffrey.blake@arm.com * associated receiver thread. The receiver thread saves the incoming packet 628733Sgeoffrey.blake@arm.com * and schedule an appropriate receive event in the event queue. 638793Sgblack@eecs.umich.edu * 642315SN/A * 3. Schedule a global barrier event periodically to keep the gem5 652315SN/A * processes in sync. 662315SN/A * Periodic barrier event to keep peer gem5 processes in sync. The basic idea 672315SN/A * is that no gem5 process can go ahead further than the simulated link 688733Sgeoffrey.blake@arm.com * transmission delay to ensure that a corresponding receive event can always 692315SN/A * be scheduled for any message coming in from a peer gem5 process. 708733Sgeoffrey.blake@arm.com * 712315SN/A * 728733Sgeoffrey.blake@arm.com * 738733Sgeoffrey.blake@arm.com * This interface is an abstract class (sendRaw() and recvRaw() 748733Sgeoffrey.blake@arm.com * methods are pure virtual). It can work with various low level 758733Sgeoffrey.blake@arm.com * send/receive service implementations (e.g. TCP/IP, MPI,...). A TCP 768733Sgeoffrey.blake@arm.com * stream socket version is implemented in dev/src/tcp_iface.[hh,cc]. 779023Sgblack@eecs.umich.edu */ 788733Sgeoffrey.blake@arm.com#ifndef __DEV_MULTI_IFACE_HH__ 798733Sgeoffrey.blake@arm.com#define __DEV_MULTI_IFACE_HH__ 808733Sgeoffrey.blake@arm.com 818733Sgeoffrey.blake@arm.com#include <array> 828733Sgeoffrey.blake@arm.com#include <mutex> 838733Sgeoffrey.blake@arm.com#include <queue> 848733Sgeoffrey.blake@arm.com#include <thread> 858733Sgeoffrey.blake@arm.com#include <utility> 868733Sgeoffrey.blake@arm.com 878733Sgeoffrey.blake@arm.com#include "dev/etherpkt.hh" 888733Sgeoffrey.blake@arm.com#include "dev/multi_packet.hh" 898733Sgeoffrey.blake@arm.com#include "sim/core.hh" 908733Sgeoffrey.blake@arm.com#include "sim/drain.hh" 918733Sgeoffrey.blake@arm.com#include "sim/global_event.hh" 928733Sgeoffrey.blake@arm.com 938733Sgeoffrey.blake@arm.comclass EventManager; 948733Sgeoffrey.blake@arm.com 958733Sgeoffrey.blake@arm.com/** 968733Sgeoffrey.blake@arm.com * The interface class to talk to peer gem5 processes. 978733Sgeoffrey.blake@arm.com */ 988733Sgeoffrey.blake@arm.comclass MultiIface : public Drainable 998733Sgeoffrey.blake@arm.com{ 1008733Sgeoffrey.blake@arm.com public: 1018733Sgeoffrey.blake@arm.com /*! 1028733Sgeoffrey.blake@arm.com * The possible reasons a multi sync among gem5 peers is needed for. 1038733Sgeoffrey.blake@arm.com */ 1048733Sgeoffrey.blake@arm.com enum 1058733Sgeoffrey.blake@arm.com class SyncTrigger { 1068733Sgeoffrey.blake@arm.com periodic, /*!< Regular periodic sync. This can be interrupted by a 1078733Sgeoffrey.blake@arm.com checkpoint sync request */ 1088733Sgeoffrey.blake@arm.com ckpt, /*!< sync before taking a checkpoint */ 1098733Sgeoffrey.blake@arm.com atomic /*!< sync that cannot be interrupted (e.g. sync at startup) */ 1108733Sgeoffrey.blake@arm.com }; 1118733Sgeoffrey.blake@arm.com 1128733Sgeoffrey.blake@arm.com private: 1138733Sgeoffrey.blake@arm.com typedef MultiHeaderPkt::MsgType MsgType; 1148733Sgeoffrey.blake@arm.com 1158733Sgeoffrey.blake@arm.com /** Sync State-Machine 1168733Sgeoffrey.blake@arm.com \dot 1178733Sgeoffrey.blake@arm.com digraph Sync { 1188733Sgeoffrey.blake@arm.com node [shape=box, fontsize=10]; 1198733Sgeoffrey.blake@arm.com idle -> busy 1208733Sgeoffrey.blake@arm.com [ label="new trigger\n by run()" fontsize=8 ]; 1219023Sgblack@eecs.umich.edu busy -> busy 1228733Sgeoffrey.blake@arm.com [ label="new message by progress():\n(msg == SyncAck &&\nwaitNum > 1) || \n(msg==CkptSyncReq &&\ntrigger == ckpt)" fontsize=8 ]; 1238733Sgeoffrey.blake@arm.com busy -> idle 1248733Sgeoffrey.blake@arm.com [ label="new message by progress():\n(msg == SyncAck &&\nwaitNum == 1)" fontsize=8 ]; 1258733Sgeoffrey.blake@arm.com busy -> interrupted 1268733Sgeoffrey.blake@arm.com [ label="new message by progress():\n(msg == CkptSyncReq &&\ntrigger == periodic)" fontsize=8 ]; 1278733Sgeoffrey.blake@arm.com idle -> asyncCkpt 1282315SN/A [ label="new message by progress():\nmsg == CkptSyncReq" fontsize=8 ]; 1292315SN/A asyncCkpt -> asyncCkpt 1302315SN/A [ label="new message by progress():\nmsg == CkptSyncReq" fontsize=8 ]; 1318733Sgeoffrey.blake@arm.com asyncCkpt -> busy 1328733Sgeoffrey.blake@arm.com [ label="new trigger by run():\ntrigger == ckpt" fontsize=8 ]; 1338733Sgeoffrey.blake@arm.com asyncCkpt -> idle 1348733Sgeoffrey.blake@arm.com [ label="new trigger by run():\n(trigger == periodic &&\nwaitNum == 0) " fontsize=8 ]; 1358733Sgeoffrey.blake@arm.com asyncCkpt -> interrupted 1368733Sgeoffrey.blake@arm.com [ label="new trigger by run():\n(trigger == periodic &&\nwaitNum > 0) " fontsize=8 ]; 1378733Sgeoffrey.blake@arm.com interrupted -> interrupted 1388733Sgeoffrey.blake@arm.com [ label="new message by progress():\n(msg == CkptSyncReq &&\nwaitNum > 1)" fontsize=8 ]; 1398733Sgeoffrey.blake@arm.com interrupted -> idle 1408733Sgeoffrey.blake@arm.com [ label="new message by progress():\n(msg == CkptSyncReq &&\nwaitNum == 1)" fontsize=8 ]; 1418733Sgeoffrey.blake@arm.com } 1428733Sgeoffrey.blake@arm.com \enddot 1432332SN/A */ 1442332SN/A /** @class Sync 1452332SN/A * This class implements global sync operations among gem5 peer processes. 1462332SN/A * 1472332SN/A * @note This class is used as a singleton object (shared by all MultiIface 1482315SN/A * objects). 1492315SN/A */ 1508733Sgeoffrey.blake@arm.com class Sync 1518733Sgeoffrey.blake@arm.com { 1522315SN/A private: 1532315SN/A /*! 1542315SN/A * Internal state of the sync singleton object. 1552315SN/A */ 1562315SN/A enum class SyncState { 1572315SN/A busy, /*!< There is an on-going sync. */ 1582315SN/A interrupted, /*!< An on-going periodic sync was interrupted. */ 1592315SN/A asyncCkpt, /*!< A checkpoint (sim_exit) is already scheduled */ 1602315SN/A idle /*!< There is no active sync. */ 1612315SN/A }; 1622315SN/A /** 1632315SN/A * The lock to protect access to the MultiSync object. 1642315SN/A */ 1658733Sgeoffrey.blake@arm.com std::mutex lock; 1668733Sgeoffrey.blake@arm.com /** 1672315SN/A * Condition variable for the simulation thread to wait on 1682315SN/A * until all receiver threads completes the current global 1692315SN/A * synchronisation. 1702315SN/A */ 1712315SN/A std::condition_variable cv; 1722315SN/A /** 1732315SN/A * Number of receiver threads that not yet completed the current global 1742315SN/A * synchronisation. 1752315SN/A */ 1762315SN/A unsigned waitNum; 1772315SN/A /** 1782315SN/A * The trigger for the most recent sync. 1792315SN/A */ 1802315SN/A SyncTrigger trigger; 1818733Sgeoffrey.blake@arm.com /** 1828733Sgeoffrey.blake@arm.com * Map sync triggers to request messages. 1838733Sgeoffrey.blake@arm.com */ 1848733Sgeoffrey.blake@arm.com std::array<MsgType, 3> triggerToMsg = {{ 1858733Sgeoffrey.blake@arm.com MsgType::cmdPeriodicSyncReq, 1868733Sgeoffrey.blake@arm.com MsgType::cmdCkptSyncReq, 1878733Sgeoffrey.blake@arm.com MsgType::cmdAtomicSyncReq 1882354SN/A }}; 1898733Sgeoffrey.blake@arm.com 1902354SN/A /** 1912332SN/A * Current sync state. 1922332SN/A */ 1932332SN/A SyncState state; 1942315SN/A 1958733Sgeoffrey.blake@arm.com public: 1968733Sgeoffrey.blake@arm.com /** 1978733Sgeoffrey.blake@arm.com * Core method to perform a full multi sync. 1988733Sgeoffrey.blake@arm.com * 1998733Sgeoffrey.blake@arm.com * @param t Sync trigger. 2008733Sgeoffrey.blake@arm.com * @param sync_tick The tick the sync was expected to happen at. 2018733Sgeoffrey.blake@arm.com * @return true if the sync completed, false if it was interrupted. 2028733Sgeoffrey.blake@arm.com * 2038733Sgeoffrey.blake@arm.com * @note In case of an interrupted periodic sync, sync_tick can be less 2042315SN/A * than curTick() when we resume (i.e. re-run) it 2052315SN/A */ 2062315SN/A bool run(SyncTrigger t, Tick sync_tick); 2072315SN/A /** 2082315SN/A * Callback when the receiver thread gets a sync message. 2092683SN/A */ 2108888Sgeoffrey.blake@arm.com void progress(MsgType m); 2118888Sgeoffrey.blake@arm.com 2128888Sgeoffrey.blake@arm.com Sync() : waitNum(0), state(SyncState::idle) {} 2132315SN/A ~Sync() {} 2142332SN/A }; 2152332SN/A 2162332SN/A 2172315SN/A /** 2188733Sgeoffrey.blake@arm.com * The global event to schedule peridic multi sync. It is used as a 2198733Sgeoffrey.blake@arm.com * singleton object. 2202315SN/A * 2218733Sgeoffrey.blake@arm.com * The periodic synchronisation works as follows. 2222315SN/A * 1. A MultisyncEvent is scheduled as a global event when startup() is 2232315SN/A * called. 2242332SN/A * 2. The progress() method of the MultisyncEvent initiates a new barrier 2258733Sgeoffrey.blake@arm.com * for each simulated Ethernet links. 2268733Sgeoffrey.blake@arm.com * 3. Simulation thread(s) then waits until all receiver threads 2272732SN/A * completes the ongoing barrier. The global sync event is done. 2282315SN/A */ 2292315SN/A class SyncEvent : public GlobalSyncEvent 2302315SN/A { 2312315SN/A public: 2322315SN/A /** 2332315SN/A * Flag to indicate that the most recent periodic sync was interrupted 2342315SN/A * (by a checkpoint request). 2358733Sgeoffrey.blake@arm.com */ 2362315SN/A bool interrupted; 2372315SN/A /** 2382315SN/A * The tick when the most recent periodic synchronisation was scheduled 2392332SN/A * at. 2408733Sgeoffrey.blake@arm.com */ 2418733Sgeoffrey.blake@arm.com Tick scheduledAt; 2422332SN/A /** 2438733Sgeoffrey.blake@arm.com * Flag to indicate an on-going drain cycle. 2448733Sgeoffrey.blake@arm.com */ 2458733Sgeoffrey.blake@arm.com bool isDraining; 2462332SN/A 2479023Sgblack@eecs.umich.edu public: 2489023Sgblack@eecs.umich.edu /** 2498733Sgeoffrey.blake@arm.com * Only the firstly instanstiated MultiIface object will 2508733Sgeoffrey.blake@arm.com * call this constructor. 2518733Sgeoffrey.blake@arm.com */ 2528733Sgeoffrey.blake@arm.com SyncEvent() : GlobalSyncEvent(Default_Pri, 0), interrupted(false), 2538733Sgeoffrey.blake@arm.com scheduledAt(0), isDraining(false) {} 2548733Sgeoffrey.blake@arm.com 2558887Sgeoffrey.blake@arm.com ~SyncEvent() { assert (scheduled() == false); } 2568733Sgeoffrey.blake@arm.com /** 2578733Sgeoffrey.blake@arm.com * Schedule the first periodic sync event. 2588733Sgeoffrey.blake@arm.com * 2598832SAli.Saidi@ARM.com * @param start Start tick for multi synchronisation 2602679SN/A * @param repeat Frequency of multi synchronisation 2612315SN/A * 2628733Sgeoffrey.blake@arm.com */ 2632315SN/A void start(Tick start, Tick repeat); 2648733Sgeoffrey.blake@arm.com /** 2658733Sgeoffrey.blake@arm.com * Reschedule (if necessary) the periodic sync event. 2668733Sgeoffrey.blake@arm.com * 2678733Sgeoffrey.blake@arm.com * @param start Start tick for multi synchronisation 2688733Sgeoffrey.blake@arm.com * @param repeat Frequency of multi synchronisation 2698733Sgeoffrey.blake@arm.com * 2708733Sgeoffrey.blake@arm.com * @note Useful if we have multiple MultiIface objects with 2718733Sgeoffrey.blake@arm.com * different 'start' and 'repeat' values for global sync. 2728733Sgeoffrey.blake@arm.com */ 2738733Sgeoffrey.blake@arm.com void adjust(Tick start, Tick repeat); 2742315SN/A /** 2758733Sgeoffrey.blake@arm.com * This is a global event so process() will be called by each 2768733Sgeoffrey.blake@arm.com * simulation threads. (See further comments in the .cc file.) 2772315SN/A */ 2788733Sgeoffrey.blake@arm.com void process() M5_ATTR_OVERRIDE; 2798733Sgeoffrey.blake@arm.com /** 2808733Sgeoffrey.blake@arm.com * Schedule periodic sync when resuming from a checkpoint. 2818733Sgeoffrey.blake@arm.com */ 2828733Sgeoffrey.blake@arm.com void resume(); 2838733Sgeoffrey.blake@arm.com 2848733Sgeoffrey.blake@arm.com void serialize(const std::string &base, CheckpointOut &cp) const; 2858733Sgeoffrey.blake@arm.com void unserialize(const std::string &base, CheckpointIn &cp); 2868733Sgeoffrey.blake@arm.com }; 2878733Sgeoffrey.blake@arm.com 2888733Sgeoffrey.blake@arm.com /** 2898733Sgeoffrey.blake@arm.com * The receive thread needs to store the packet pointer and the computed 2908733Sgeoffrey.blake@arm.com * receive tick for each incoming data packet. This information is used 2918949Sandreas.hansson@arm.com * by the simulation thread when it processes the corresponding receive 2928733Sgeoffrey.blake@arm.com * event. (See more comments at the implemetation of the recvThreadFunc() 2938733Sgeoffrey.blake@arm.com * and RecvPacketIn() methods.) 2948733Sgeoffrey.blake@arm.com */ 2958733Sgeoffrey.blake@arm.com typedef std::pair<EthPacketPtr, Tick> RecvInfo; 2968733Sgeoffrey.blake@arm.com 2978733Sgeoffrey.blake@arm.com /** 2988733Sgeoffrey.blake@arm.com * Comparison predicate for RecvInfo, needed by the recvQueue. 2998733Sgeoffrey.blake@arm.com */ 3008733Sgeoffrey.blake@arm.com struct RecvInfoCompare { 3018733Sgeoffrey.blake@arm.com bool operator()(const RecvInfo &lhs, const RecvInfo &rhs) 3028733Sgeoffrey.blake@arm.com { 3038733Sgeoffrey.blake@arm.com return lhs.second > rhs.second; 3048733Sgeoffrey.blake@arm.com } 3058733Sgeoffrey.blake@arm.com }; 3068733Sgeoffrey.blake@arm.com 3078733Sgeoffrey.blake@arm.com /** 3088733Sgeoffrey.blake@arm.com * Customized priority queue used to store incoming data packets info by 3098733Sgeoffrey.blake@arm.com * the receiver thread. We need to expose the underlying container to 3108733Sgeoffrey.blake@arm.com * enable iterator access for serializing. 3118733Sgeoffrey.blake@arm.com */ 3128733Sgeoffrey.blake@arm.com class RecvQueue : public std::priority_queue<RecvInfo, 3138733Sgeoffrey.blake@arm.com std::vector<RecvInfo>, 3148733Sgeoffrey.blake@arm.com RecvInfoCompare> 3158733Sgeoffrey.blake@arm.com { 3169023Sgblack@eecs.umich.edu public: 3178733Sgeoffrey.blake@arm.com std::vector<RecvInfo> &impl() { return c; } 3188733Sgeoffrey.blake@arm.com const std::vector<RecvInfo> &impl() const { return c; } 3198733Sgeoffrey.blake@arm.com }; 3208733Sgeoffrey.blake@arm.com 3219023Sgblack@eecs.umich.edu /* 3228733Sgeoffrey.blake@arm.com * The priority queue to store RecvInfo items ordered by receive ticks. 3239023Sgblack@eecs.umich.edu */ 3248733Sgeoffrey.blake@arm.com RecvQueue recvQueue; 3258733Sgeoffrey.blake@arm.com /** 3268733Sgeoffrey.blake@arm.com * The singleton Sync object to perform multi synchronisation. 3278733Sgeoffrey.blake@arm.com */ 3288733Sgeoffrey.blake@arm.com static Sync *sync; 3298733Sgeoffrey.blake@arm.com /** 3308733Sgeoffrey.blake@arm.com * The singleton SyncEvent object to schedule periodic multi sync. 3318733Sgeoffrey.blake@arm.com */ 3328733Sgeoffrey.blake@arm.com static SyncEvent *syncEvent; 3338733Sgeoffrey.blake@arm.com /** 3348733Sgeoffrey.blake@arm.com * Tick to schedule the first multi sync event. 3358733Sgeoffrey.blake@arm.com * This is just as optimization : we do not need any multi sync 3368733Sgeoffrey.blake@arm.com * event until the simulated NIC is brought up by the OS. 3378733Sgeoffrey.blake@arm.com */ 3388733Sgeoffrey.blake@arm.com Tick syncStart; 3398733Sgeoffrey.blake@arm.com /** 3408733Sgeoffrey.blake@arm.com * Frequency of multi sync events in ticks. 3418733Sgeoffrey.blake@arm.com */ 3428733Sgeoffrey.blake@arm.com Tick syncRepeat; 3438733Sgeoffrey.blake@arm.com /** 3448733Sgeoffrey.blake@arm.com * Receiver thread pointer. 3452323SN/A * Each MultiIface object must have exactly one receiver thread. 3462315SN/A */ 3479023Sgblack@eecs.umich.edu std::thread *recvThread; 3489023Sgblack@eecs.umich.edu /** 3492315SN/A * The event manager associated with the MultiIface object. 3508733Sgeoffrey.blake@arm.com */ 3518733Sgeoffrey.blake@arm.com EventManager *eventManager; 3528733Sgeoffrey.blake@arm.com 3532323SN/A /** 3548733Sgeoffrey.blake@arm.com * The receive done event for the simulated Ethernet link. 3552679SN/A * It is scheduled by the receiver thread for each incoming data 3562323SN/A * packet. 3572323SN/A */ 3588733Sgeoffrey.blake@arm.com Event *recvDone; 3592323SN/A 3602315SN/A /** 3618733Sgeoffrey.blake@arm.com * The packet that belongs to the currently scheduled recvDone event. 3628733Sgeoffrey.blake@arm.com */ 3638733Sgeoffrey.blake@arm.com EthPacketPtr scheduledRecvPacket; 3642679SN/A 3652315SN/A /** 3662315SN/A * The link delay in ticks for the simulated Ethernet link. 3672315SN/A */ 3682315SN/A Tick linkDelay; 3698733Sgeoffrey.blake@arm.com 3708733Sgeoffrey.blake@arm.com /** 3718733Sgeoffrey.blake@arm.com * The rank of this process among the gem5 peers. 3728733Sgeoffrey.blake@arm.com */ 3738733Sgeoffrey.blake@arm.com unsigned rank; 3748733Sgeoffrey.blake@arm.com /** 3758733Sgeoffrey.blake@arm.com * Total number of receiver threads (in this gem5 process). 3768733Sgeoffrey.blake@arm.com * During the simulation it should be constant and equal to the 3778733Sgeoffrey.blake@arm.com * number of MultiIface objects (i.e. simulated Ethernet 3788733Sgeoffrey.blake@arm.com * links). 3798733Sgeoffrey.blake@arm.com */ 3808733Sgeoffrey.blake@arm.com static unsigned recvThreadsNum; 3818733Sgeoffrey.blake@arm.com /** 3822315SN/A * The very first MultiIface object created becomes the master. We need 3838733Sgeoffrey.blake@arm.com * a master to co-ordinate the global synchronisation. 3848733Sgeoffrey.blake@arm.com */ 3858733Sgeoffrey.blake@arm.com static MultiIface *master; 3868733Sgeoffrey.blake@arm.com 3872315SN/A protected: 3888733Sgeoffrey.blake@arm.com /** 3898733Sgeoffrey.blake@arm.com * Low level generic send routine. 3908733Sgeoffrey.blake@arm.com * @param buf buffer that holds the data to send out 3918733Sgeoffrey.blake@arm.com * @param length number of bytes to send 3928733Sgeoffrey.blake@arm.com * @param dest_addr address of the target (simulated NIC). This may be 3938733Sgeoffrey.blake@arm.com * used by a subclass for optimization (e.g. optimize broadcast) 3948733Sgeoffrey.blake@arm.com */ 3958733Sgeoffrey.blake@arm.com virtual void sendRaw(void *buf, 3968733Sgeoffrey.blake@arm.com unsigned length, 3978733Sgeoffrey.blake@arm.com const MultiHeaderPkt::AddressType dest_addr) = 0; 3988733Sgeoffrey.blake@arm.com /** 3992315SN/A * Low level generic receive routine. 4002315SN/A * @param buf the buffer to store the incoming message 4012315SN/A * @param length buffer size (in bytes) 4028733Sgeoffrey.blake@arm.com */ 4032315SN/A virtual bool recvRaw(void *buf, unsigned length) = 0; 4048887Sgeoffrey.blake@arm.com /** 4058887Sgeoffrey.blake@arm.com * Low level request for synchronisation among gem5 processes. Only one 4068887Sgeoffrey.blake@arm.com * MultiIface object needs to call this (in each gem5 process) to trigger 4078887Sgeoffrey.blake@arm.com * a multi sync. 4088887Sgeoffrey.blake@arm.com * 4098887Sgeoffrey.blake@arm.com * @param sync_req Sync request command. 4108887Sgeoffrey.blake@arm.com * @param sync_tick The tick when sync is expected to happen in the sender. 4112315SN/A */ 4128733Sgeoffrey.blake@arm.com virtual void syncRaw(MsgType sync_req, Tick sync_tick) = 0; 4132315SN/A /** 4142315SN/A * The function executed by a receiver thread. 4158793Sgblack@eecs.umich.edu */ 4168793Sgblack@eecs.umich.edu void recvThreadFunc(); 4178793Sgblack@eecs.umich.edu /** 4188793Sgblack@eecs.umich.edu * Receive a multi header packet. Called by the receiver thread. 4198793Sgblack@eecs.umich.edu * @param header the structure to store the incoming header packet. 4208793Sgblack@eecs.umich.edu * @return false if any error occured during the receive, true otherwise 4218793Sgblack@eecs.umich.edu * 4228809Sgblack@eecs.umich.edu * A header packet can carry a control command (e.g. 'barrier leave') or 4238793Sgblack@eecs.umich.edu * information about a data packet that is following the header packet 4248793Sgblack@eecs.umich.edu * back to back. 4258809Sgblack@eecs.umich.edu */ 4268793Sgblack@eecs.umich.edu bool recvHeader(MultiHeaderPkt::Header &header); 4278793Sgblack@eecs.umich.edu /** 4288809Sgblack@eecs.umich.edu * Receive a data packet. Called by the receiver thread. 4298809Sgblack@eecs.umich.edu * @param data_header The packet descriptor for the expected incoming data 4308793Sgblack@eecs.umich.edu * packet. 4312315SN/A */ 4322315SN/A void recvData(const MultiHeaderPkt::Header &data_header); 4332332SN/A 4342315SN/A public: 4352315SN/A 4362315SN/A /** 4372332SN/A * ctor 4382332SN/A * @param multi_rank Rank of this gem5 process within the multi run 4392315SN/A * @param sync_start Start tick for multi synchronisation 4402315SN/A * @param sync_repeat Frequency for multi synchronisation 4412315SN/A * @param em The event manager associated with the simulated Ethernet link 4428733Sgeoffrey.blake@arm.com */ 4438733Sgeoffrey.blake@arm.com MultiIface(unsigned multi_rank, 4442315SN/A Tick sync_start, 4452315SN/A Tick sync_repeat, 4462315SN/A EventManager *em); 4472315SN/A 4482315SN/A virtual ~MultiIface(); 4492354SN/A /** 4502315SN/A * Send out an Ethernet packet. 4512315SN/A * @param pkt The Ethernet packet to send. 4528733Sgeoffrey.blake@arm.com * @param send_delay The delay in ticks for the send completion event. 4532315SN/A */ 4548733Sgeoffrey.blake@arm.com void packetOut(EthPacketPtr pkt, Tick send_delay); 4552315SN/A /** 4562315SN/A * Fetch the next packet from the receive queue. 4572315SN/A */ 4582315SN/A EthPacketPtr packetIn(); 4598733Sgeoffrey.blake@arm.com 4602315SN/A /** 4618733Sgeoffrey.blake@arm.com * spawn the receiver thread. 4622315SN/A * @param recv_done The receive done event associated with the simulated 4632315SN/A * Ethernet link. 4642315SN/A * @param link_delay The link delay for the simulated Ethernet link. 4658733Sgeoffrey.blake@arm.com */ 4662315SN/A void spawnRecvThread(Event *recv_done, 4678733Sgeoffrey.blake@arm.com Tick link_delay); 4682315SN/A /** 4698733Sgeoffrey.blake@arm.com * Initialize the random number generator with a different seed in each 4708733Sgeoffrey.blake@arm.com * peer gem5 process. 4718733Sgeoffrey.blake@arm.com */ 4722315SN/A void initRandom(); 4732332SN/A 4747823Ssteve.reinhardt@amd.com DrainState drain() M5_ATTR_OVERRIDE; 4752315SN/A 4762732SN/A /** 4772315SN/A * Callback when draining is complete. 4782315SN/A */ 4792315SN/A void drainDone(); 4809023Sgblack@eecs.umich.edu 4819023Sgblack@eecs.umich.edu /** 4829023Sgblack@eecs.umich.edu * Initialize the periodic synchronisation among peer gem5 processes. 4832315SN/A */ 4842315SN/A void startPeriodicSync(); 4852315SN/A 4868733Sgeoffrey.blake@arm.com void serialize(const std::string &base, CheckpointOut &cp) const; 4872315SN/A void unserialize(const std::string &base, CheckpointIn &cp); 4888733Sgeoffrey.blake@arm.com 4892315SN/A}; 4908733Sgeoffrey.blake@arm.com 4918733Sgeoffrey.blake@arm.com 4928733Sgeoffrey.blake@arm.com#endif 4932732SN/A