perf-stats - Compare profiling runs

s6 perf-stats reads metrics.json profiling output, summarizes per-event durations, and can compare one run against another.

Usage

# Show the latest run under ./logs, or fall back to ./metrics.json
s6 perf-stats

# Compare up to two run directories or explicit metrics files
s6 perf-stats ./logs/runs/2024_10_01 ./logs/runs/2024_10_02

# Save a Markdown report
s6 perf-stats ./logs/runs/2024_10_01 -o perf.md

Behavior

  • With no positional arguments, it searches ./logs recursively for the most recently modified metrics.json. If nothing is found, it uses ./metrics.json.

  • Each positional argument may be either a file or a directory. Directory inputs are searched recursively for metrics.json, then fall back to a direct metrics.json child.

  • Only one or two runs are supported.

  • The command uses s6.utils.profiler.process_frames() and s6.utils.profiler.compute_statistics() to compute count, total_ms, min_ms, max_ms, average_ms, std_dev_ms, variance_ms, median_ms, p5_ms, p25_ms, p75_ms, p95_ms, iqr_ms, best_5pct_avg_ms, and worst_5pct_avg_ms.

  • With two runs, the table includes delta and % improvement columns and marks improvements or regressions with color in the terminal and spans in the Markdown output.

  • With -o/--output, the command writes a Markdown report grouped by event.