#
14209:7efe1c187149 |
|
22-Apr-2016 |
Andreas Sandberg <andreas.sandberg@arm.com> |
stats: Add beta support for HDF5 stat dumps
This changeset add support for stat dumps in the HDF5 file format. HDF5 is a binary data format that represents data in a file-system-like balanced tree. It has native support for N-dimensional arrays and binary data (e.g., frame buffers).
It has the following benefits over traditional text stat files:
* Efficient storage of time series (multiple stat dumps)
* Fast lookup of stats
* Plenty of existing tooling (e.g., Python libraries and graphical viewers)
* File format can be used to store frame buffers together with normal stats.
Drawbacks:
* Large startup cost (single stat dump larger than text equivalent)
* Stat dumps are slower than text
Known limitations:
* Distributions and histograms aren't supported.
HDF5 stat output can be enabled using the 'h5' URL scheme when overriding the stat file name on gem5's command line. The following parameters are supported:
* chunking (unsigned): Number of time steps to pre-allocate (default: 10)
* desc (bool): Output stat descriptions (default: True)
* formulas (bool): Output derived stats (default: True)
Example gem5 command line:
./build/ARM/gem5.opt \ --stats-file="h5://stats.h5?desc=False;formulas=False" \ configs/example/fs.py
Example Python stat consumer that computes IPC: import h5py
f = h5py.File('stats.h5', 'r') group = f['/system/cpu'] for i, c in zip(group['committedInsts'], group['numCycles']): print i, c, i / c
Change-Id: I351c6cbff2fb7bef9012f47876ba227ed288975b Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com> Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/8121 Tested-by: kokoro <noreply+kokoro@google.com> Reviewed-by: Jason Lowe-Power <jason@lowepower.com> Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br> Reviewed-by: Ciro Santilli <ciro.santilli@arm.com>
|
#
14205:197360deaa20 |
|
26-Jun-2019 |
Andreas Sandberg <andreas.sandberg@arm.com> |
stats: Add support for hierarchical stats
This change makes the stat system aware of the hierarchical nature of stats. The aim is to achieve the following goals:
* Make the SimObject hierarchy explicit in the stat system (i.e., get rid of name() + ".foo"). This makes stat naming less fragile and makes it possible to implement hierarchical formats like XML/HDF5/JSON in a clean way.
* Make it more convenient to split stats into a separate struct/class that can be bound to a SimObject. This makes the namespace cleaner and makes stat accesses a bit more obvious.
* Make it possible to build groups of stats in C++ that can be used in subcomponents in a SimObject (similar to what we do for checkpoint sections). This makes it easier to structure large components.
* Enable partial stat dumps. Some of our internal users have been asking for this since a full stat dump can be large.
* Enable better stat access from Python.
This changeset implements solves the first three points by introducing a class (Stats::Group) that owns statistics belonging to the same object. SimObjects inherit from Stats::Group since they typically have statistics.
New-style statistics need to be associated with a parent group at instantiation time. Instantiation typically sets the name and the description, other parameters need to be set by overriding Group::regStats() just like with legacy stats. Simple objects with scalar stats can typically avoid implementing regStats() altogether since the stat name and description are both specified in the constructor.
For convenience reasons, statistics groups can be merged into other groups. This means that a SimObject can create a stat struct that inherits from Stats::Group and merge it into the parent group (SimObject). This can make the code cleaner since statistics tracking gets grouped into a single object.
Stat visitors have a new API to expose the group structure. The Output::beginGroup(name) method is called at the beginning of a group and the Output::endGroup() method is called when all stats, and sub-groups, have been visited. Flat formats (e.g., the text format) typically need to maintain a stack to track the full path to a stat.
Legacy, flat, statistics are still supported after applying this change. These stats don't belong to any group and stat visitors will not see a Output::beginGroup(name) call before their corresponding Output::visit() methods are called.
Change-Id: I9025d61dfadeabcc8ecf30813ab2060def455648 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/19368 Tested-by: kokoro <noreply+kokoro@google.com> Reviewed-by: Jason Lowe-Power <jason@lowepower.com> Reviewed-by: Daniel Carvalho <odanrc@yahoo.com.br>
|
#
11988:665cd5f8b52b |
|
27-Feb-2017 |
Andreas Sandberg <andreas.sandberg@arm.com> |
python: Use PyBind11 instead of SWIG for Python wrappers
Use the PyBind11 wrapping infrastructure instead of SWIG to generate wrappers for functionality that needs to be exported to Python. This has several benefits:
* PyBind11 can be redistributed with gem5, which means that we have full control of the version used. This avoid a large number of hard-to-debug SWIG issues we have seen in the past.
* PyBind11 doesn't rely on a custom C++ parser, instead it relies on wrappers being explicitly declared in C++. The leads to slightly more boiler-plate code in manually created wrappers, but doesn't doesn't increase the overall code size. A big benefit is that this avoids strange compilation errors when SWIG doesn't understand modern language features.
* Unlike SWIG, there is no risk that the wrapper code incorporates incorrect type casts (this has happened on numerous occasions in the past) since these will result in compile-time errors.
As a part of this change, the mechanism to define exported methods has been redesigned slightly. New methods can be exported either by declaring them in the SimObject declaration and decorating them with the cxxMethod decorator or by adding an instance of PyBindMethod/PyBindProperty to the cxx_exports class variable. The decorator has the added benefit of making it possible to add a docstring and naming the method's parameters.
The new wrappers have the following known issues:
* Global events can't be memory managed correctly. This was the case in SWIG as well.
Change-Id: I88c5a95b6cf6c32fa9e1ad31dfc08b2e8199a763 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Andreas Hansson <andreas.hansson@arm.com> Reviewed-by: Andrew Bardsley <andrew.bardsley@arm.com> Reviewed-on: https://gem5-review.googlesource.com/2231 Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com> Reviewed-by: Pierre-Yves PĂ©neau <pierre-yves.peneau@lirmm.fr> Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
|