Remote Runner: Runner Log Statistics

GtRrRunnerLogTimingAnalyzer Object << #GtRrRunnerLogTimingAnalyzer slots: { #filename . #signals . #timingStats . #currentRun }; tag: 'JobStats'; package: 'RemoteRunner' provides basic functionality for storing and analysing statistics from the remote runner log file, facilitating:

monitoring performance over a timespan

helping identify the cause of delays in job execution

There are basically two phases:

1. Gathering and storing statistics

2. Comparing an individual run to the norm.

Only basic statistics are stored, i.e. average, min, max and standard deviation, so all logs that are fed in to the statistics need to be comparible, i.e. if the characteristics of the system, speed of workers, time for workers to start, etc. normally vary a lot, the comparisons will be less useful.

In all cases below the log files are those stored by GtRemoteRunner Object << #GtRemoteRunner traits: {TGtWithAnnouncementsQueue}; slots: { #port . #server . #jobStartupAndShutdown . #managerStartupAndShutdown . #availableWorkers . #currentlyExecutingJobs . #currentlyExecutingTasks . #sharedAccess . #registeredWorkers . #taskTimeout . #workerTimeout . #jobTimeout . #timeoutManager . #announcer . #runnerStats . #log . #taskQueue . #changesSyncEvents . #announcementNotifier . #logEmitter }; tag: 'Runner'; package: 'RemoteRunner' , typically specified using GtRemoteRunner>>#moveLoggingTo: moveLoggingTo: aFileReference "Replace the receiver's logger with one that logs to the supplied file" | fileReference oldLogger newLogger | fileReference := aFileReference asFileReference resolve. (log isNil or: [ log fileReference isNotNil and: [ log fileReference resolve = fileReference resolve ] ]) ifTrue: [ ^ self ]. oldLogger := log. newLogger := SessionFallbackFileLogger new serializer: GtBeaconJsonSignalSerializer new; filename: fileReference. log := StreamLoggerFlusher new logger: newLogger. self startLogger: log. oldLogger stop; close. ^ oldLogger. .

Gathering and storing statistics

Creating new baseline statistics

For a new dataset, analyse a number of files and save the timing stats, e.g.:

analyzer := GtRrRunnerLogTimingAnalyzer analyzeFiles: 
	('C:\gt\akg\new' asFileReference childrenMatching: 'remoteRunner-*.log').
newStats := analyzer timingStats.
newStats saveToFile: 'remoteRunner-baselineStats.ston'.
  

Loading baseline stats

A previously saved dataset can be loaded with:

baselineStats := GtRrRunnerLogTimingStats loadFromFile: 'remoteRunnner-baselineStats.ston'.
  

Adding new datapoints to a dataset

The baseline dataset can be extended with:

"Assuming that the remoteRunner log files are newer than those above"
analyzer := GtRrRunnerLogTimingAnalyzer analyzeFiles: 
	('C:\gt\akg\new' asFileReference childrenMatching: 'remoteRunner-*.log').
analyzer appendStatsTo: baselineStats.
baselineStats saveToFile: 'remoteRunner-baselineStats.ston'.
  

Comparing a single run to the baseline stats

To obtain a report inspect:

"Create comparator and run comparison"
comparator := GtRrRunnerLogTimingStatsComparator 
	baseline: baselineStats 
	testLogFile: 'remoteRunner.log' asFileReference.
comparator compare.
comparator.