GemStone compatibility and development rules
All GT / GemStone code related operations should be performed on the live system using available tools (via the gtoolkit mcp server). git repositories should only be inspected / modified for shell scripts, topaz files or if specifically required.
All code is managed from within GT, any time a method is to be added or modified it should be done in GT and then updated in GemStone.
The compile tools run through their own machinery (e.g. GtLMethodCompilationDataExtractor
), so changing that code through the tool changes the tool part-way through a batch. Add new callees before their callers, in separate calls, and never put a caller and its new callee in the same compileMethods batch. If the tool breaks itself, only an evaluation can repair it.
When modifying an existing method with compileMethods, gemstoneCompileMethod or gemstoneCompileMethods, OMIT protocol: the method keeps its current protocol (in GemStone, its category). Don't pass a protocol you've guessed or copied. A wrong protocol is a silent change, and for an EXTENSION METHOD (where the protocol begins with a '*') it moves the method to another package. Change the protocol of an existing method only when explicitly asked. If you feel the need to change a protocol, stop and ask for confirmation.
Give a protocol for every NEW method. A new method without one goes into 'as yet unclassified'. A blank or whitespace-only protocol counts as no protocol, and a given protocol is trimmed.
To check a protocol, use (Class >> #selector) protocolName in GT and Class categoryOfSelector: #selector environmentId: 0 in the stone.
GemStone will commit the value of class and class-instance variables to the database. In general, this is not what is desired when porting code. In particular, class methods that answer a default, instance or uniqueInstance that store the instance in a class or class-instance variable should not store the value in the database. In these cases they should be added to SessionTemps with a name based on the class and accessor method, e.g. SessionTemps current at: #Class_selector_value ifAbsentPut: [ self new ].
There are many differences between GT and GemStone classes. Where GemStone requires an alternate implementation there are a few different potential solutions:
Adding a pragma with the <gsCode: gemstoneSourceString> pragma to the method will result in the gemstoneSourceString being used when the code is exported (see GtGemStoneGtCodeExporter
). An example is GtGemStoneTopazExporterExamples>>#gsCodeExampleMethod
.
NOTE: the gemstoneSourceString replaces everything below the pragma, meaning that the method signature doesn't need to be included, so in the example linked above the method signature and comment will be part of the generated GemStone code.
Note that the method's class must be included in the GtGemStoneGtCodeExporter>>#gsCodeClasses
list.
This can only be used in methods owned by feenkcom, i.e. CompiledMethod>>#hasFeenkRepositoryAvailable
answers true.
Extension methods can be added to GemStone with the code managed by GT by using GtGemStoneExtensionMethod
and GtGemStoneExtensionMethods
, see their class comments for further information.
This is generally preferred for Pharo methods that aren't managed by GT, e.g. Collection>>#intersection:
, and where the class otherwise isn't exported.
For methods that are largely the same in both GT and GemStone, Object>>#gtDo:gemstoneDo:
provides a way to have a small section of code be platform specific. An example use is in BlockClosureOnProcessExamples>>#getCallStack
.
When choosing between gsCode: and gtDo:gemstoneDo:, gsCode: is normally used when the method body is significantly different between GT and GemStone.
The following are a number of commonly encountered issues when getting GT code working in GemStone:
In GemStone, new, and its variants, don't call initialize.
Unless the GT version also defines new, or one of its variants, this will be resolved by adding a GtGemStoneExtensionMethod
method.
When debugging code being ported from GT to GemStone, if an existing method appears to not initialise instance variables, first check that the initialize method is being called prior to adding lazy initialisation to the ported method.
Navigating the class / method structure in GemStone is often much slower than in GT. Instead of requesting classes directly, lookups such as superclasses, subclasses, references, etc. should be made through the ClassOrganizer gtDefault instance.
The following list of methods that exist in GT, but are missing in GemStone and have equivalents in GemStone that are considered close enough for now. The list is formatted originalMethod -> replacementMethod: 'canPerform: aSelector' -> 'canUnderstand: aSelector' 'includesSubstring: aString' -> 'findPattern: { aString} startingAt: 1' 'includesSubstring: aString caseSensitive: false' -> 'findPatternNoCase: { aString } startingAt: 1'
Package extension methods on classes that are native in GemStone (e.g. Dictionary, ZnMessage, the Zinc HTTP classes) are silently left out of GT.gs. Add a GtGemStoneExtensionMethods
descriptor for each one. Before a load, run GtGemStoneGtCodeExporter new staticExportAnalysisForPackageNames: on the changed packages to find them.
A descriptor that replaces a method that already exists in the stone must use hash: with the sha256 of the native source (unix line endings, trimRight, UTF-8; see sha256Of: in zinc/zinc-methods.topaz). hash: #new means the method must not exist, and the load fails with 'Method already exists but extension requires a new method'. Check #new descriptors against the native sources, not with gemstoneGetCodeOfMethods (see Testing in the stone).
Wide strings aren't under String: DoubleByteString and QuadByteString are MultiByteString < CharacterCollection. A String-only extension misses them, e.g. when JSON parsed from non-Latin-1 text produces DoubleByteStrings. Add the descriptor on CharacterCollection.
Methods that use thisContext, reference Smalltalk, or contain a literal line starting with % are listed in GtGemStoneGtCodeExporter>>#excludedMethodsList
and are not exported. Exclusion overrides gsCode:, so when a gsCode: variant is added the method must also be removed from that list. Excluding a method that comes from a trait removes it from every user of the flattened trait.
Topaz reads a line starting with % as the end of the method, even inside a string literal (the literal '%' on its own line included). Build the character instead, e.g. String with: $%, as in GtLResponseMarkdownFormat>>#initialize
.
Document why a method is excluded, or is GT only, in the method's own comment. GT reformats source on compile and moves comments inside a { } brace array to after the array, so put the explanation in the leading comment.
A stub superclass (e.g. in GToolkit-Stubs) gives its subclasses no inherited protocol, and the failures are often indirect. Export the real class as an additional class instead.
A Pharo WriteStream subclass that only wraps another stream can't subclass GemStone's WriteStream, which writes into its own collection without going through nextPut:. Use an Object alternate class definition and add the stream protocol it needs with extension method descriptors (e.g. ZnIgnoreNewLineWriterStream).
Object>>asString answers a description such as 'aDictionary', not the print string as in Pharo, and Collection>>printString is truncated at 700 characters. Shared code that turns arbitrary objects into text should use printOn: on a stream (String streamContents: [ :s | anObject printOn: s ]).
Don't rely on a Symbol and a String with the same characters being interchangeable as dictionary keys. Normalise keys, or compare with sameContentAs:. GToolkit4Json is forgiving on input and writes String keys on output.
The stone runs in legacy string comparison mode. decodeFromUTF8 and utf8Decoded answer Unicode7/Unicode16, and comparing those with traditional strings (e.g. the Set includes: in ZnHeaders>>normalizeHeaderKey:) raises 'non-Unicode argument disallowed in Unicode comparison'. Use decodeFromUTF8ToString, which answers String or DoubleByteString. Don't change the stone's comparison mode: it is a stone-wide setting.
substrings: keeps empty tokens, unlike Pharo.
The GsDevKit GZipReadStream fails with MessageNotUnderstood #squeakReplaceFrom:to:with:startingAt:. Don't alias that selector to replaceFrom:to:with:startingAt:: inflating relies on a forward, element-by-element copy within one collection, and an overlap-safe copy decodes silently wrong. Use the native ByteArray>>_decompress: 1 (1 = gzip header, 0 = raw deflate) in a gsCode: variant, as in GtLModelExamplesExecutor>>#decodeHistory:
.
The stone's Zinc HTTP classes (ZnClient, ZnResponse, ZnUtils, ...) are the frozen native ones in ZincGlobals, older than GT's. Only ZnUrl exists twice: the native one in ZincGlobals and GT's in Globals. String>>asZnUrl deliberately answers the native one, because the native ZnClient consumes it. Fill API gaps in the native classes with extension method descriptors.
The native Zinc has no TLS stream; GtGsSecureSocketStream
provides one by upgrading the connected socket with GsSecureSocket newClientFromGsSocket:. GsSecureSocket class>>setCaCertLocation silently turns verification off when it finds no CA bundle, so set the bundle explicitly and fail if it is missing, and call setExpectedHost: for host-name checks.
The native ZnStringEntity uses a null encoder when no charset is given, which corrupts non-ASCII bodies. JSON entities must declare charset=utf-8.
Runtime configuration (e.g. an API key, a client setting) must not be stored in class variables or default instances such as ZnNetworkingUtils default, as they are committed. Use SessionTemps (see Class and class-instance variables).
The stone has no AsyncFuture runtime. Gt4LlmCore provides GtLSynchronousFuture
and GtLSynchronousPromise
, with ExecBlock>>asAsyncFuture and asAsyncPromise bridges: the future evaluates its block on await, and the execution configuration is ignored. Code that sends asAsyncFuture / await: runs unchanged.
An error raised inside an announcement handler ends up as a rejected promise, so an example that waits on a semaphore times out (60 s) instead of failing. In GtLChat a nil response usually hides the real error in the reply's finalMessage (a GtLErrorMessage with its exception); look there first.
Run examples in the stone with GtGemStoneExampleRunner
: add the classes to GtGemStoneExampleRunner>>#select
, run (GtGemStoneExampleRunner on: GtGemStoneSessionRegistry defaultSession) run, then inspect summary and notPassed. All existing tests and examples must keep passing.
gemstoneGetCodeOfMethods and the other session tools show the GT session's uncommitted compiles, not what a fresh load of GT.gs produces, and uncommitted compiles can be lost. At milestones, export GT.gs and load it into a clean stone (loadGsGt.sh, log in $GT_CURRENT_DIR/loadGsGt.log) before trusting a result.
gsCode: variants run only in the stone, so GT examples don't cover them. When you change one, make sure an example class that exercises it is in GtGemStoneExampleRunner>>#select
, or check it by hand in the stone. The Magritte compile-tool example classes aren't in select at present.
The runner selects every example in its target classes, so the totals grow as examples are added. Compare a result with the per-class list from the last baseline, not with the bare counts.
When responding, GT method references are normally styled using the gtMethod: attribute. As GemStone methods often don't exist in the GT image, they should be styled using the normal code styling, i.e. back-ticks (`).