From cf3ff3f98277b043c6c84f4cbba7fdc1b44808a6 Mon Sep 17 00:00:00 2001 From: Johan Maasing Date: Sat, 19 Apr 2025 19:24:45 +0200 Subject: [PATCH] Documentation and merge behaviour for state files --- README.md | 106 +++++++++++++++--- .../dsl/ast/StatesVisitorTransformer.java | 44 ++++++-- .../java/nu/zoom/dsl/cli/EndpointsCLI.java | 8 +- states-templates/Codecs.scala.ftl | 19 ++++ states-templates/Types.scala.ftl | 23 ++++ test01.states | 25 ++++- 6 files changed, 193 insertions(+), 32 deletions(-) create mode 100644 states-templates/Codecs.scala.ftl create mode 100644 states-templates/Types.scala.ftl diff --git a/README.md b/README.md index 2293eef..2e73f92 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,17 @@ parser and a code generator using [freemarker](https://freemarker.apache.org). | mytemplate.xxx.ftl | \____________________\ ``` + +Endgen currently contains two separate parsers: +* endpoint - A DSL for expressing HTTP endpoints. +* state - A DSL for expressing state and transitions. + +Which parser that is used to read the input file is determined by the file name ending +'.endpoints' or '.states' or by a command line argument. + +The endpoint DSL and the state DSL share the grammar for expressing configuration and data types +,see below for details. + ## How to Run You need a Java 21 (or later) runtime and java in the path. A very convenient way to install a java runtime is [SdkMan](https://sdkman.io). @@ -39,20 +50,23 @@ Unpack the archive, run the provided shellscript file. ### Usage ``` -Usage: run.sh [-hvV] [-o=] [-t=] +sage: run.sh [-hvV] [-o=] [-p=] [-t=] Generate source code from an endpoints specification file. The source endpoints DSL file. -h, --help Show this help message and exit. -o, --output= The directory to write the generated code to. - Default is ~/endpoints-output + Default is endpoints-output + -p, --parser= Force use of a specific parser instead of + determining from filename. Valid values: + Endpoints, States. -t, --template= The template directory. Default is - ~/endpoints-templates + endpoints-template -v, --verbose Print verbose debug messages. -V, --version Print version information and exit. ``` -## DSL example +## Endpoint DSL example In the simplest form the DSL looks like this ``` /some/endpoint <- SomeType(foo:String) @@ -80,6 +94,8 @@ This is the ANTLR grammar for the root of the DSL ```antlrv4 document : generatorconfig? (namedTypeDeclaration|endpoint)* ; ``` + +### Configuration block Meaning that the DSL file has an optional `generatorconfig` block at the top. Then you can write either; a type definition, or an endpoint declaration, as many times as you like. @@ -99,7 +115,9 @@ Embedded(foo:Bar) This consists of a config block with 2 items, the 'package' and the 'mykey' definition. These are available to be used in the freemarker template as a Map of String-keys to String-values. -`/some/endpoint <- SomeType(foo:String)` is an endpoint declaration. It declares one endpoint that have a request body +### Endpoint definition + +`/some/endpoint <- SomeType(foo:String)` is an endpoint definition. It declares one endpoint that have a request body data type called `SomeType` that has a field called `foo` of the type `String`. ### Data types @@ -110,7 +128,7 @@ just named field-name and the other string is named field-type. `Embedded(foo:Bar)` is a `namedTypeDeclaration` which is parsed the same way as the request type above. But isn't tied to a specific endpoint. -### Automatically named data types +### Automatically named endpoint data types `/some/other/endpoint <- (bar:Seq[Embedded])` is another endpoint declaration. However this time the request body is not named in the DSL. But all datatypes must have a name so it will simply name it after the last path segment and tack on the string 'Request' at the end. So the AST till contain a datatype named `endpointRequest` with a field named @@ -122,7 +140,7 @@ decide to generate in the templates. The only 'semantic' validation the parser performs is to check that not two types have the same name. -### Reponse data types +### Endpoint reponse data type It is possible to have an optional response data type declared like so: `/some/other/endpoint <- (bar:Seq[Embedded]) -> ResponseType(foo: Bar)` @@ -130,9 +148,45 @@ It is possible to have an optional response data type declared like so: The right pointing arrow `->` denotes a response type, it can be an anonymous data type in which case the parser till name it from the last path segment and add 'Response' to the end of the data type name. -### DSL config -The only key in the config block the generator looks at is called `ending`, this will be used as the file ending for -the resulting file of applying the freemarker template. +### State grammar + +This is an example of a state file: +``` +start -> middle: message, +middle -> middle: selfmessage, +middle -> end: endmessage +``` + +It contains 3 state definitions `start`, `middle` and `end`. A state definition will be parsed as a data type with +the name of the state as the type name. + +It also contains 3 message definitions `message`, `selfmessage` and `endmessage`. Message definitions will also be +parsed as data types. + +Since the parser will extract datatypes it is possible to define the fields of the data types. This is a slightly more +complicated example: + +``` +start(foo:Foo) -> middle: message(a: String), +middle(bar:Bar) -> middle: selfmessage, +middle -> end: endmessage +``` +Where for example the data type for `middle` will have the field declaration with the name `bar` and the type `Bar`. + +Fields for the same state data type, or message data type, will be merged. Here is a complex example: + +``` +start(s:S) -> middle(foo:foo): message(foo:foo), +middle -> middle(bar:bar): selfmessage(bar:bar), +middle -> end: message(bar:baz) +``` + +Not that we can declare fields on both the `from` and `to` state declarations. The `middle` datat type will have field +definitons for `foo` and `bar`. + +The data type for `message` will have fields for `foo` and `bar`. + +One restriction is that states and messages may not have the same name, i.e. be parsed as the same data type. ## Generating If the parser is successful it will hold the following data in the AST @@ -140,12 +194,17 @@ If the parser is successful it will hold the following data in the AST ```java public record DocumentNode( Map config, - List typeDefinitions, - List endpoints) { + Set typeDefinitions, + List endpoints, + Set states) { } ``` -This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your freemarker template like this: +Depending on the parser used the endpoints or the states will be null but config and typeDefinitions are populated the +same for both parsers. + +This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your +freemarker template like this: ```injectedfreemarker <#list typeDefinitions as type> @@ -153,7 +212,7 @@ This will be passed to the freemarker engine as the 'root' data object, meaning ``` -That is, you can directly reference `typeDefinitions`, `endpoints` or `config`. +That is, you can directly reference `typeDefinitions`, `endpoints`, `states` or `config`. ### Config The config object is simply a String-map with the keys and values unfiltered from the input file. Here is an example @@ -212,4 +271,21 @@ Output data type. -``` \ No newline at end of file +``` + +### States + +The set of states will hold items of this shape: + +```injectedfreemarker +public record StateNode(String name, String data, Set transitions) { +} +``` + +and the transitions has this structure: + +```injectedfreemarker +public record TransitionNode(String message, String toState) { +} +``` + diff --git a/parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java b/parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java index e1360a7..cd4bef0 100644 --- a/parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java +++ b/parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java @@ -62,23 +62,49 @@ public class StatesVisitorTransformer extends StatesBaseVisitor getTypes() { - // TODO calculate data types from NodeTypes and MessageTypes with duplicate check. - HashMap typeNodes = new HashMap<>(); + final HashMap stateTypeNodes = new HashMap<>(); this.nodeTypes.forEach(typeNode -> { - if (typeNodes.containsKey(typeNode.name())) { - throw new RuntimeException("Duplicate type name: " + typeNode.name()); + if (stateTypeNodes.containsKey(typeNode.name())) { + TypeNode mergedNode = mergeTypeFields(typeNode, stateTypeNodes.get(typeNode.name())); + stateTypeNodes.put(typeNode.name(), mergedNode); } else { - typeNodes.put(typeNode.name(), typeNode); + stateTypeNodes.put(typeNode.name(), typeNode); } }) ; + final HashMap messageTypeNodes = new HashMap<>(); this.messageTypes.forEach(typeNode -> { - if (typeNodes.containsKey(typeNode.name())) { - throw new RuntimeException("Duplicate type name: " + typeNode.name()); + if (stateTypeNodes.containsKey(typeNode.name())) { + throw new ParseException("Message " + typeNode.name() + " conflicts with state with the same type name"); + } + if (messageTypeNodes.containsKey(typeNode.name())) { + TypeNode mergedNode = mergeTypeFields(typeNode, messageTypeNodes.get(typeNode.name())); + messageTypeNodes.put(typeNode.name(), mergedNode); } else { - typeNodes.put(typeNode.name(), typeNode); + messageTypeNodes.put(typeNode.name(), typeNode); } }) ; - return Set.of(typeNodes.values().toArray(new TypeNode[0])) ; + HashSet allTypeNodes = new HashSet<>(stateTypeNodes.values()); + allTypeNodes.addAll(messageTypeNodes.values()); + return allTypeNodes ; + } + + private TypeNode mergeTypeFields(TypeNode t1, TypeNode t2) { + List t1Fields = (t1 != null) ? t1.fields() : List.of() ; + List t2Fields = (t2 != null) ? t2.fields() : List.of() ; + HashMap mergedFields = new HashMap<>(); + t1Fields.forEach(field -> { + if (mergedFields.containsKey(field.name())) { + throw new ParseException("Duplicate field name: " + field.name()); + } + mergedFields.put(field.name(), field); + }); + t2Fields.forEach(field -> { + if (mergedFields.containsKey(field.name())) { + throw new ParseException("Duplicate field name: " + field.name()); + } + mergedFields.put(field.name(), field); + }); + return new TypeNode(t1.name(), mergedFields.values().stream().toList()) ; } private List extractFields(StatesParser.TypeDeclarationContext declaration) { diff --git a/parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java b/parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java index 724d9b4..e3d5794 100644 --- a/parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java +++ b/parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java @@ -43,12 +43,12 @@ public class EndpointsCLI implements Callable { private Path file; @SuppressWarnings("CanBeFinal") - @Option(names = {"-t", "--template"}, description = "The template directory. Default is ~/endpoints-templates") - private Path templateDir = Paths.get(System.getProperty("user.dir"), "endpoints-templates"); + @Option(names = {"-t", "--template"}, defaultValue = "endpoints-template", description = "The template directory. Default is ${DEFAULT-VALUE}") + private Path templateDir ; @SuppressWarnings("CanBeFinal") - @Option(names = {"-o", "--output"}, description = "The directory to write the generated code to. Default is ~/endpoints-output") - private Path outputDir = Paths.get(System.getProperty("user.dir"), "endpoints-output"); + @Option(names = {"-o", "--output"}, defaultValue = "endpoints-output", description = "The directory to write the generated code to. Default is ${DEFAULT-VALUE}") + private Path outputDir ; @SuppressWarnings("unused") @Option(names = {"-v", "--verbose"}, description = "Print verbose debug messages.") diff --git a/states-templates/Codecs.scala.ftl b/states-templates/Codecs.scala.ftl new file mode 100644 index 0000000..f11070f --- /dev/null +++ b/states-templates/Codecs.scala.ftl @@ -0,0 +1,19 @@ +// Copyright 2025 "Johan Maasing" +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package ${config.package} + +object Codecs: +<#list typeDefinitions as type> + given Codec[${type.name?cap_first}] = deriveCodec + diff --git a/states-templates/Types.scala.ftl b/states-templates/Types.scala.ftl new file mode 100644 index 0000000..f2322b6 --- /dev/null +++ b/states-templates/Types.scala.ftl @@ -0,0 +1,23 @@ +// Copyright 2025 "Johan Maasing" +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package ${config.package} + +object StateAndMessageTypes: +<#list typeDefinitions?sort as type> + case class ${type.name?cap_first}( + <#list type.fields as field> + ${field.name} : ${field.type}, + + ) + diff --git a/test01.states b/test01.states index d1f78a9..bcb304f 100644 --- a/test01.states +++ b/test01.states @@ -1,4 +1,21 @@ -{ title: SomeNodes } -start(s:S) -> middle: message(foo:bar), -middle -> middle: selfmessage, -middle -> end: endmessage(bar:baz) \ No newline at end of file +/* + Copyright 2025 "Johan Maasing" + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +{ title: SomeNodes, package: nu.zoom.dsl.states } + +start(s:S) -> middle(foo:foo): message(foo:foo), +middle -> middle(bar:bar): selfmessage(bar:bar), +middle -> end: message(bar:baz)