History log of /gem5/util/stats/stats.py
Revision Date Author Comments
# 13540:da30e62884ee 10-Jan-2019 Andrea Mondelli <andrea.mondelli@ucf.edu>

misc: updated shabang for python script

The default python on MacOS doesn’t have an alias to python2.
The official python version supported in gem5 is Python2.7.

This patch updates the shabang according to the version required in gem5.

Change-Id: I9533c0f7858b5b3cab0ef101be1ee5cd718105b0
Reviewed-on: https://gem5-review.googlesource.com/c/15375
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>


# 11828:36b064696175 10-Feb-2017 Jason Lowe-Power <jason@lowepower.com>

misc: Update #!env calls for python to explicit version

In some newer Linux distributions, env python default to Python 3.0. This
patch explicitly uses "python2" instead of just "python" for all scripts
that use #!

Reported-by: Sanchayan Maity <maitysanchayan@gmail.com>
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>


# 6277:f67c9be3cd57 02-Jul-2009 Nathan Binkert <nate@binkert.org>

stats: fix usage of jobfile


# 2716:b9114064d77a 11-Jun-2006 Nathan Binkert <binkertn@umich.edu>

Merge iceaxe.:/Volumes/work/research/m5/head
into iceaxe.:/Volumes/work/research/m5/merge

src/cpu/simple/base.cc:
src/kern/kernel_stats.cc:
src/kern/kernel_stats.hh:
src/kern/system_events.cc:
src/kern/system_events.hh:
src/python/m5/objects/System.py:
src/sim/system.cc:
src/sim/system.hh:
hand merge


# 2665:a124942bacb8 31-May-2006 Ali Saidi <saidi@eecs.umich.edu>

Updated Authors from bk prs info


# 2343:a2b4a6ccee56 10-Jun-2006 Nathan Binkert <binkertn@umich.edu>

Remove all binning stuff


# 2187:494da52c4eac 26-Feb-2006 Nathan Binkert <binkertn@umich.edu>

fix some minor stats stuff

util/stats/stats.py:
fix up a few of the stats.


# 2006:3ca085495c69 22-Nov-2005 Nathan Binkert <binkertn@umich.edu>

Major improvements in the graph output code. Mostly adding more
options, making existing options more visible and dealing with
holes in data better.

util/stats/barchart.py:
- move the options for BarChart to a base class ChartOptions so
they can be more easily set and copied.
- add an option to set the chart size (so you can adjust the aspect ratio)
- don't do the add_subplot thing, use add_axes directly so we can
affect the size of the figure itself to make room for the legend
- make the initial array bottom floating point so we don't lose precision
- add an option to set the limits on the y axis
- use a figure legend instead of an axes legend so we can put the legend
outside of the actual chart. Also add an option to set the fontsize of
the legend.
- initial hack at outputting csv files
util/stats/db.py:
don't print out an error when the run is missing from the database
just return None, the error will be print elsewhere.
util/stats/output.py:
- make StatOutput derive from ChartOptions so that it's easier to
set default chart options.
- make the various output functions (graph, display, etc.) take the
name of the data as a parameter instead of making it a parameter to
__init__. This allows me to create the StatOutput object with
generic parameters while still being able to specialize the name
after the fact
- add support for graph_group and graph_bars to be applied to multiple
configuration groups. This results in a cross product of the groups
to be generated and used.
- flush the html file output as we go so that we can load the file
while graphs are still being generated.
- make the proxy a parameter to the graph function so the proper system's
data can be graphed
- for any groups or bars that are completely missing, remove them from
the graph. This way, if we decide not to do a set of runs, there won't
be holes in the data.
- output eps and ps by default in addition to the png.
util/stats/profile.py:
- clean up the data structures that are used to store the function
profile information and try our best to avoid keeping extra data
around that isn't used.
- make get() return None if a job is missing so we know it was
missing rather than the all zeroes thing.
- make the function profile categorization stuff total up to 100%
- Fixup the x-axis and y-axis labels.
- fix the dot file output stuff.
util/stats/stats.py:
support the new options stuff for StatOutput


# 2005:5d2963051cc7 22-Nov-2005 Nathan Binkert <binkertn@umich.edu>

Change the set of statistics that are generated when specifying "all"


# 2002:7167e8c3ef31 22-Nov-2005 Nathan Binkert <binkertn@umich.edu>

make the NUMCPUS parameter always refer to the total number of cpus
on the test system.
add an option for pio_delay_write to run.py

util/stats/stats.py:
full0 -> run0 due to run.py change
sim_ticks doesn't make sense with tick = ps, so use
one of the cpu's numCycles paramter


# 1958:144ef1d5315f 02-Nov-2005 Ali Saidi <saidi@eecs.umich.edu>

Merge zizzer:/bk/m5
into zeep.eecs.umich.edu:/z/saidi/work/m5


# 1957:895f4795130f 02-Nov-2005 Ali Saidi <saidi@eecs.umich.edu>

Change the output of stability info a little bit


# 1929:fb189519cb06 21-Oct-2005 Nathan Binkert <binkertn@umich.edu>

Major cleanup of the statistics handling code

util/stats/db.py:
Build a result object as the result of a query operation so it is
easier to populate and contains a bit more information than just
a big dict. Also change the next level data into a matrix instead
of a dict of dicts.
Move the "get" function into the Database object. (The get function
is used by the output parsing function as the interface for accessing
backend storage, same interface for profile stuff.)
Change the old get variable to the method variable, it describes how
the get works, (whether using sum, stdev, etc.)
util/stats/display.py:
Clean up the display functions, mostly formatting.
Handle values the way they should be now.
util/stats/info.py:
Totally re-work how values are accessed from their data store.
Access individual values on demand instead of calculating everything
and passing up a huge result from the bottom.
This impacts the way that proxying works, and in general, everything
is now esentially a proxy for the lower level database. Provide new
operators: unproxy, scalar, vector, value, values, total, and len which
retrieve the proper result from the object they are called on.
Move the ProxyGroup stuff (proxies of proxies!) here from the now gone
proxy.py file and integrate the shared parts of the code. The ProxyGroup
stuff allows you to write formulas without specifying the statistics
until evaluation time.

Get rid of global variables!
util/stats/output.py:
Move the dbinfo stuff into the Database itself. Each source should
have it's own get() function for accessing it's data.
This get() function behaves a bit differently than before in that it
can return vectors as well, deal with these vectors and with no result
conditions better.
util/stats/stats.py:
the info module no longer has the source global variable, just
create the database source and pass it around as necessary


# 1918:0c4f083eccc8 18-Oct-2005 Nathan Binkert <binkertn@umich.edu>

Default jobfile for stats

util/stats/stats.py:
Make the default jobfile Test.py in the current directory
add the -J flag to tell it not to use a jobfile


# 1881:fc205a7edd58 17-Sep-2005 Nathan Binkert <binkertn@umich.edu>

Totally re-work the way that jobfiles are done so there is more
information that can be used for other aspects of sending jobs.
New graphing output stuff with matplotlib.

util/pbs/job.py:
Shuffle code around and create the JobDir class which encapsulates
all of the functionality needed for making, organizing, and cleaning
a job directory.
Better status output
util/pbs/jobfile.py:
Majory re-working of the jobfile code.
A job file now consists of several objects that describe how
jobs should be run, it includes information about checkpoints,
and graphing.
util/pbs/send.py:
use the new jobfile code.
deal with the 15 character limit of pbs by truncating the name and
using the raj hack.
util/stats/db.py:
fix the __str__ function for nodes
provide __getitem__ for the Database class
util/stats/stats.py:
use the jobfile stuff to figure out what the proper naming
and organziation of the graphs should be.
move all output code to output.py, get rid of ploticus and use
matplotlib


# 1772:a3a83e812a5e 05-Jun-2005 Nathan Binkert <binkertn@umich.edu>

Cleanup copyright stuff. Add our copyright files that
are ours

arch/alpha/alpha_linux_process.hh:
arch/alpha/alpha_tru64_process.hh:
base/loader/object_file.cc:
base/loader/object_file.hh:
sim/process.cc:
sim/process.hh:
remove $Id$ string
cpu/ozone/cpu.cc:
cpu/ozone/cpu_impl.hh:
cpu/ozone/ea_list.cc:
cpu/ozone/ea_list.hh:
kern/linux/sched.hh:
kern/linux/thread_info.hh:
Add M5 Copyright
cpu/trace/opt_cpu.cc:
dev/rtcreg.h:
nit
kern/linux/aligned.hh:
kern/linux/hwrpb.hh:
util/oprofile-top.py:
util/stats/db.py:
util/stats/dbinit.py:
util/stats/display.py:
util/stats/info.py:
util/stats/print.py:
util/stats/stats.py:
Cleanup copyright


# 1758:74acd5b23964 05-Jun-2005 Ali Saidi <saidi@eecs.umich.edu>

added copyright

kern/linux/sched.hh:
kern/linux/thread_info.hh:
got rid of everything but exactly what we needed
util/categories.py:
newest version from one of my repositories


# 1621:33fdc8ebda51 06-Apr-2005 Nathan Binkert <binkertn@umich.edu>

full_system isn't a useful parameter anymore, get rid of it.

python/m5/objects/Root.mpy:
sim/universe.cc:
util/stats/stats.py:
full_system isn't a useful parameter


# 1604:c986abdfc71f 24-Mar-2005 Ron Dreslinski <rdreslin@umich.edu>

Update so that statistics can be plotted correctly

util/stats/stats.py:
Changed some stuff for graphing purposes:
full_cpu is now full0
frequencies are now s,m,f,q not s,6,8,q
L2 is now l2
etherdev is now etherdev0

May want to consider fact that NAT box should be the sum of etherdev0 and etherdev1 (not in script yet)


# 1596:d11171f51a63 23-Mar-2005 Ali Saidi <saidi@eecs.umich.edu>

update profile code to use shared categories file

util/stats/stats.py:
add database command to help


# 1331:a0b8619e0e72 19-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

added total bytes/bandwidth/packets formulas to nics
cleaned up stability code and wrote some better help for stats.py
fixed sample bug in info.py

dev/ns_gige.cc:
dev/ns_gige.hh:
dev/sinic.cc:
dev/sinic.hh:
add total bandwidth/packets/bytes stats
util/stats/info.py:
fixed samples bug
util/stats/stats.py:
cleaned up stability code and wrote a bit better help


# 1329:1e7bd1684f64 18-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

Ability to check formulas


# 1318:57d49cdb92a5 18-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

now really done with stability stats stuff


# 1317:0b6026d3000b 18-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

finished stability stats option


# 1309:7daf185ddae8 14-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

added stats code to check stability

util/stats/db.py:
added working listticks (for printing) and retticks(for using in python) code
util/stats/stats.py:
added stability function that checks if all samples are within 10% of mean.


# 1308:2078b53d65d9 14-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

automerged had issues with the options.XXXXX in stats.py


# 1307:e6b9976895c6 12-Jan-2005 Nathan Binkert <binkertn@umich.edu>

More graph output junk

util/stats/stats.py:
Add the graphing output for 6GHz and 8GHz runs


# 1301:f85f6fb43474 13-Jan-2005 Ali Saidi <saidi@eecs.umich.edu>

fix a display bug
add option to limit results to a set of ticks
fix ticks code to work

util/stats/info.py:
change samples -> ticks and pass all parameters
util/stats/stats.py:
add option to select a set of ticks and fix display bug


# 1269:419b73347a1e 18-Nov-2004 Nathan Binkert <binkertn@umich.edu>

stats python fixes for dealing with vectors better. graph tweaking
ipkb stat and formulas from the command line.

util/stats/info.py:
no need to raise an attribute error if two values aren't found
in the exact same set of runs. Would be good to check that each
run is the same though.
util/stats/stats.py:
more graph tweaking
command to execute a formula from the command line.
add interrupts per kilobyte of data


# 1209:7eb1f18fcaa6 17-Nov-2004 Nathan Binkert <binkertn@umich.edu>

Fix some commands.

util/stats/stats.py:
we only need the system if we're issuing one of the commands that
uses a stored formula.


# 1165:234f3cdd541a 15-Nov-2004 Nathan Binkert <binkertn@umich.edu>

Fix more of the stats python stuff

util/stats/info.py:
Make the binnings stuff work again.
util/stats/stats.py:
small patch for graphing
make it so we can print out bins for the stat command


# 1162:a45bd001ac39 14-Nov-2004 Nathan Binkert <binkertn@umich.edu>

update graph generating code

util/stats/stats.py:
tweak the graphing stuff for the new configurations we have.
add more graph types.
nsgige -> etherdev
deal with memory hierarchy change by using L2 instead of L3


# 1076:0e05a8c0c598 10-Sep-2004 Nathan Binkert <binkertn@umich.edu>

fix the options stuff

util/stats/stats.py:
get the options from the options struct now
gratuitously change the output directory for graphs.


# 1049:b175a798c8d4 09-Aug-2004 Nathan Binkert <binkertn@umich.edu>

Totally re-do/reorganize the python part of the statistics code
Make the database creation/removal/cleanup code use python
Make formulas work with the database
Add support to do some graphing, but needs more work
Still need to work on vectors, 2d vectors, dists and vectordists