Analysing stored GT reports

Reports are recorded so that a problem can be understood after it happened. Analysing them means three steps: find the reports you need (by type, day or image, from their metadata only), read them back into report objects, and then query those objects like any other objects. How to record and store reports is on GT reports available for data collection; how the infrastructure works is on Report infrastructure for collecting data in GT.

Finding stored reports

Inspect the store to browse its folder (#selector gtRootFolderFor: aView <gtView> ^ aView forward title: 'Reports folder'; priority: 20; object: [ rootFolder ]; view: #gtTreeFor:; actionButtonIcon: BrGlamorousVectorIcons inspect tooltip: 'Inspect the reports folder' action: [ :aButton | aButton phlow spawnObject: rootFolder ]; actionButtonIcon: BrGlamorousVectorIcons folder tooltip: 'Open the reports folder in the OS file browser' action: [ :aButton | rootFolder ensureCreateDirectory. rootFolder openInOSFileBrowser ] ). Every folder with a meta.json is one stored report; GtStoredReportFiles Object << #GtStoredReportFiles slots: { #folder . #metadata }; package: 'GToolkit-Utility-SpaceTally' #selector folder: aFolder ^ self new setFolder: aFolder reads its metadata and shows it (#selector gtMetadataFor: aView <gtView> ^ aView forward title: 'Metadata'; priority: 5; object: [ metadata ]; view: #gtSummaryFor: ), lists its files (#selector gtFilesFor: aView <gtView> ^ aView list title: 'Files'; priority: 10; items: [ folder children asArray sort: [ :a :b | a basename <= b basename ] ]; itemText: [ :each | each basename ] ), and its Read action reads the report back (#selector gtReadReportActionFor: anAction <gtAction> ^ anAction button icon: BrGlamorousVectorIcons inspect; tooltip: 'Read the report and inspect it'; action: [ :aButton | aButton phlow spawnObject: self report ] ). A folder without meta.json is an interrupted write and is not found below.

reportsStore := GtReportsLocalStore default.
  

storedReports := (reportsStore rootFolder allChildrenMatching: GtReportMetadata fileName)
	collect: [:each | GtStoredReportFiles folder: each parent].
  

The objects you work with

Object What it is Useful messages GtStoredReportFiles Object << #GtStoredReportFiles slots: { #folder . #metadata }; package: 'GToolkit-Utility-SpaceTally' one stored report folder, with its metadata already read #selector metadata ^ metadata , #selector folder ^ folder , #selector reportClass ^ Smalltalk globals at: metadata reportClassName asSymbol , #selector report ^ self reportClass fromStoredFiles: self GtReportMetadata Object << #GtReportMetadata slots: { #formatVersion . #reportType . #reportClassName . #reportAlias . #reportUid . #storageId . #storedAt . #machineName . #imageData . #files }; package: 'GToolkit-Utility-SpaceTally' the content of meta.json #selector reportType ^ reportType , #selector storedAt ^ storedAt , #selector reportUid ^ reportUid , #selector storageId ^ storageId , #selector files ^ files , #selector imageData ^ imageData , #selector formatVersion ^ formatVersion GtReportFileDescription Object << #GtReportFileDescription slots: { #name . #numberOfBytes }; package: 'GToolkit-Utility-SpaceTally' one data file listed in the metadata #selector name ^ name , #selector numberOfBytes ^ numberOfBytes the report the object that was recorded, read back from its files (e.g. GtMessageTallyReport Object << #GtMessageTallyReport slots: { #textualContent . #imageData . #reportUid . #startTime . #duration . #pollPeriodMilliseconds . #numberOfSamples . #measuredMilliseconds . #vmParametersDiff . #messageTally }; package: 'GToolkit-Utility-SpaceTally' ) its own accessors and views, and #selector imageData ^ imageData

Selecting and grouping on the metadata only reads meta.json files, so it is cheap. #selector report ^ self reportClass fromStoredFiles: self reads the data files: select first, then read. A space tally takes about a minute to read.

Example analyses

The examples use storedReports from above. Each starts from a question.

storedReports groupedBy: [:each | each metadata reportType].
  

Which reports of each type are there? From the metadata only (#selector reportType ^ reportType ).

(storedReports groupedBy: [:each | each metadata reportType])
	collect: [:eachGroup |
		(eachGroup sumNumbers: [:each |
			each metadata files sumNumbers: [:eachFile | eachFile numberOfBytes]])
				humanReadableIsqByteSize].
  

How much space does each report type take? From the file sizes in the metadata (#selector numberOfBytes ^ numberOfBytes ), without reading any report.

storedReports detectMax: [:each | each metadata storedAt].
  

Which report was stored last? (#selector storedAt ^ storedAt , in UTC)

(((storedReports select: [:each | each metadata reportType = 'message-tally'])
	collect: [:each | each report])
		select: [:each | each textualContent includesSubstring: 'BlSpace>>#processEvents'])
			sorted: [:a :b | a numberOfSamples >= b numberOfSamples].
  

Which Message Tallies saw a given method, the busiest first? A slowdown that comes back shows up in several tallies (#selector textualContent ^ textualContent , #selector numberOfSamples ^ numberOfSamples ).

((storedReports select: [:each | each metadata reportType = 'processes'])
	collect: [:each | each report])
		collect: [:eachReport |
			eachReport -> (eachReport processReports
				select: [:each | each name includesSubstring: 'Morphic'])].
  

Where was a given process in each processes report? For example the Morphic UI process (#selector processReports ^ processReports , GtProcessReport Object << #GtProcessReport slots: { #id . #name . #priority . #isQuiescent . #contextReports }; package: 'GToolkit-Utility-SpaceTally' ), with its stack (#selector stackTraceDescription ^ String streamContents: [ :aStream | self stackTraceDescriptionOn: aStream ] ).

((storedReports select: [:each | each metadata reportType = 'garbage-collects'])
	collect: [:each | each report])
		collect: [:each | each -> each recordedEventsDuration].
  

How much time did the recorded full garbage collections take in each report? (#selector recordedEventsDuration ^ events inject: 0 seconds into: [ :total :anEvent | total + anEvent duration ] )

latestSpaceTally := ((storedReports select: [:each | each metadata reportType = 'space-tally'])
	detectMax: [:each | each metadata storedAt]) report.
  

(latestSpaceTally spaceTally results asArray
	sorted: [:a :b | a spaceForInstances >= b spaceForInstances]) first: 20.
  

Which classes take the most memory in the latest space tally? Reads one space tally, which takes about a minute (#selector spaceTally ^ spaceTally , GtSpaceTallyItem Object << #GtSpaceTallyItem slots: { #analyzedPackageName . #codeSize . #analyzedClassName . #instanceCount . #spaceForInstances . #detailsPerSpace . #objectsTally }; package: 'GToolkit-Utility-SpaceTally' #selector spaceForInstances ^ spaceForInstances ).

(storedReports groupedBy: [:each | each metadata reportUid])
	select: [:eachGroup | eachGroup size > 1].
  

Was a report stored more than once? Copies share the report uid (#selector reportUid ^ reportUid ); keep one of each group to analyse every report once.