Documentation and merge behaviour for state files

README.md

@@ -32,6 +32,17 @@ parser and a code generator using [freemarker](https://freemarker.apache.org).
                                                     | mytemplate.xxx.ftl |
                                                     \____________________\
 ``` 
+
+Endgen currently contains two separate parsers: 
+* endpoint - A DSL for expressing HTTP endpoints.
+* state - A DSL for expressing state and transitions.
+
+Which parser that is used to read the input file is determined by the file name ending 
+'.endpoints' or '.states' or by a command line argument.
+
+The endpoint DSL and the state DSL share the grammar for expressing configuration and data types 
+,see below for details.
+
 ## How to Run
 You need a Java 21 (or later) runtime and java in the path. A very convenient way to install a java runtime is [SdkMan](https://sdkman.io).
 
@@ -39,20 +50,23 @@ Unpack the archive, run the provided shellscript file.
 
 ### Usage
 ```
-Usage: run.sh [-hvV] [-o=<outputDir>] [-t=<templateDir>] <file>
+sage: run.sh [-hvV] [-o=<outputDir>] [-p=<parser>] [-t=<templateDir>] <file>
 Generate source code from an endpoints specification file.
       <file>                 The source endpoints DSL file.
   -h, --help                 Show this help message and exit.
   -o, --output=<outputDir>   The directory to write the generated code to.
-                               Default is ~/endpoints-output
+                               Default is endpoints-output
+  -p, --parser=<parser>      Force use of a specific parser instead of
+                               determining from filename. Valid values:
+                               Endpoints, States.
   -t, --template=<templateDir>
                              The template directory. Default is
-                               ~/endpoints-templates
+                               endpoints-template
   -v, --verbose              Print verbose debug messages.
   -V, --version              Print version information and exit.
 ```
 
-## DSL example
+## Endpoint DSL example
 In the simplest form the DSL looks like this
 ```
 /some/endpoint <- SomeType(foo:String)
@@ -80,6 +94,8 @@ This is the ANTLR grammar for the root of the DSL
 ```antlrv4
 document                : generatorconfig? (namedTypeDeclaration|endpoint)* ;
 ```
+
+### Configuration block
 Meaning that the DSL file has an optional `generatorconfig` block at the top. Then you can write either; a type 
 definition, or an endpoint declaration, as many times as you like.
 
@@ -99,7 +115,9 @@ Embedded(foo:Bar)
 This consists of a config block with 2 items, the 'package' and the 'mykey' definition. These are available to be used
 in the freemarker template as a Map of String-keys to String-values.
 
-`/some/endpoint <- SomeType(foo:String)` is an endpoint declaration. It declares one endpoint that have a request body 
+### Endpoint definition
+
+`/some/endpoint <- SomeType(foo:String)` is an endpoint definition. It declares one endpoint that have a request body 
 data type called `SomeType` that has a field called `foo` of the type `String`.
 
 ### Data types
@@ -110,7 +128,7 @@ just named field-name and the other string is named field-type.
 `Embedded(foo:Bar)` is a `namedTypeDeclaration` which is parsed the same way as the request type above. But isn't tied
 to a specific endpoint.
 
-### Automatically named data types
+### Automatically named endpoint data types
 `/some/other/endpoint <- (bar:Seq[Embedded])` is another endpoint declaration. However this time the request body is
 not named in the DSL. But all datatypes must have a name so it will simply name it after the last path segment and
 tack on the string 'Request' at the end. So the AST till contain a datatype named `endpointRequest` with a field named
@@ -122,7 +140,7 @@ decide to generate in the templates.
 
 The only 'semantic' validation the parser performs is to check that not two types have the same name.
 
-### Reponse data types
+### Endpoint reponse data type
 It is possible to have an optional response data type declared like so:
 
 `/some/other/endpoint <- (bar:Seq[Embedded]) -> ResponseType(foo: Bar)`
@@ -130,9 +148,45 @@ It is possible to have an optional response data type declared like so:
 The right pointing arrow `->` denotes a response type, it can be an anonymous data type in which case the parser till 
 name it from the last path segment and add 'Response' to the end of the data type name.
 
-### DSL config
+### State grammar
-The only key in the config block the generator looks at is called `ending`, this will be used as the file ending for
+
-the resulting file of applying the freemarker template.
+This is an example of a state file:
+```
+start -> middle: message,
+middle -> middle: selfmessage,
+middle -> end: endmessage
+```
+
+It contains 3 state definitions `start`, `middle` and `end`. A state definition will be parsed as a data type with
+the name of the state as the type name. 
+
+It also contains 3 message definitions `message`, `selfmessage` and `endmessage`. Message definitions will also be 
+parsed as data types.
+
+Since the parser will extract datatypes it is possible to define the fields of the data types. This is a slightly more
+complicated example:
+
+```
+start(foo:Foo) -> middle: message(a: String),
+middle(bar:Bar) -> middle: selfmessage,
+middle -> end: endmessage
+```
+Where for example the data type for `middle` will have the field declaration with the name `bar` and the type `Bar`.
+
+Fields for the same state data type, or message data type, will be merged. Here is a complex example:
+
+```
+start(s:S) -> middle(foo:foo): message(foo:foo),
+middle -> middle(bar:bar): selfmessage(bar:bar),
+middle -> end: message(bar:baz)
+```
+
+Not that we can declare fields on both the `from` and `to` state declarations. The `middle` datat type will have field
+definitons for `foo` and `bar`. 
+
+The data type for `message` will have fields for `foo` and `bar`.
+
+One restriction is that states and messages may not have the same name, i.e. be parsed as the same data type.
 
 ## Generating
 If the parser is successful it will hold the following data in the AST
@@ -140,12 +194,17 @@ If the parser is successful it will hold the following data in the AST
 ```java
 public record DocumentNode(
         Map<String, String> config,
-        List<TypeNode> typeDefinitions,
+        Set<TypeNode> typeDefinitions,
-        List<EndpointNode> endpoints) {
+        List<EndpointNode> endpoints,
+        Set<StateNode> states) {
 }
 ```
 
-This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your freemarker template like this:
+Depending on the parser used the endpoints or the states will be null but config and typeDefinitions are populated the 
+same for both parsers.
+
+This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your 
+freemarker template like this:
 
 ```injectedfreemarker
 <#list typeDefinitions as type>
@@ -153,7 +212,7 @@ This will be passed to the freemarker engine as the 'root' data object, meaning
 </#list>
 ```
 
-That is, you can directly reference `typeDefinitions`, `endpoints` or `config`.
+That is, you can directly reference `typeDefinitions`, `endpoints`, `states` or `config`.
 
 ### Config
 The config object is simply a String-map with the keys and values unfiltered from the input file. Here is an example
@@ -213,3 +272,20 @@ Output data type.
 
 </#list>
 ```
+
+### States
+
+The set of states will hold items of this shape:
+
+```injectedfreemarker
+public record StateNode(String name, String data, Set<TransitionNode> transitions) {
+}
+```
+
+and the transitions has this structure:
+
+```injectedfreemarker
+public record TransitionNode(String message, String toState) {
+}
+```
+

parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java

@@ -62,23 +62,49 @@ public class StatesVisitorTransformer extends StatesBaseVisitor<StatesParser.Doc
     }
 
     public Set<TypeNode> getTypes() {
-        // TODO calculate data types from NodeTypes and MessageTypes with duplicate check.
+        final HashMap<String, TypeNode> stateTypeNodes = new HashMap<>();
-        HashMap<String, TypeNode> typeNodes = new HashMap<>();
         this.nodeTypes.forEach(typeNode -> {
-            if (typeNodes.containsKey(typeNode.name())) {
+            if (stateTypeNodes.containsKey(typeNode.name())) {
-                throw new RuntimeException("Duplicate type name: " + typeNode.name());
+                TypeNode mergedNode = mergeTypeFields(typeNode, stateTypeNodes.get(typeNode.name()));
+                stateTypeNodes.put(typeNode.name(), mergedNode);
             } else {
-                typeNodes.put(typeNode.name(), typeNode);
+                stateTypeNodes.put(typeNode.name(), typeNode);
             }
         }) ;
+        final HashMap<String, TypeNode> messageTypeNodes = new HashMap<>();
         this.messageTypes.forEach(typeNode -> {
-            if (typeNodes.containsKey(typeNode.name())) {
+            if (stateTypeNodes.containsKey(typeNode.name())) {
-                throw new RuntimeException("Duplicate type name: " + typeNode.name());
+                throw new ParseException("Message " + typeNode.name() + " conflicts with state with the same type name");
+            }
+            if (messageTypeNodes.containsKey(typeNode.name())) {
+                TypeNode mergedNode = mergeTypeFields(typeNode, messageTypeNodes.get(typeNode.name()));
+                messageTypeNodes.put(typeNode.name(), mergedNode);
             } else {
-                typeNodes.put(typeNode.name(), typeNode);
+                messageTypeNodes.put(typeNode.name(), typeNode);
             }
         }) ;
-        return Set.of(typeNodes.values().toArray(new TypeNode[0])) ;
+        HashSet<TypeNode> allTypeNodes = new HashSet<>(stateTypeNodes.values());
+        allTypeNodes.addAll(messageTypeNodes.values());
+        return allTypeNodes ;
+    }
+
+    private TypeNode mergeTypeFields(TypeNode t1, TypeNode t2) {
+        List<FieldNode> t1Fields = (t1 != null) ? t1.fields() : List.of() ;
+        List<FieldNode> t2Fields = (t2 != null) ? t2.fields() : List.of() ;
+        HashMap<String, FieldNode> mergedFields = new HashMap<>();
+        t1Fields.forEach(field -> {
+            if (mergedFields.containsKey(field.name())) {
+                throw new ParseException("Duplicate field name: " + field.name());
+            }
+            mergedFields.put(field.name(), field);
+        });
+        t2Fields.forEach(field -> {
+            if (mergedFields.containsKey(field.name())) {
+                throw new ParseException("Duplicate field name: " + field.name());
+            }
+            mergedFields.put(field.name(), field);
+        });
+        return new TypeNode(t1.name(), mergedFields.values().stream().toList()) ;
     }
 
     private List<FieldNode> extractFields(StatesParser.TypeDeclarationContext declaration) {

parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java

@@ -43,12 +43,12 @@ public class EndpointsCLI implements Callable<Integer> {
 	private Path file;
 
 	@SuppressWarnings("CanBeFinal")
-    @Option(names = {"-t", "--template"}, description = "The template directory. Default is ~/endpoints-templates")
+    @Option(names = {"-t", "--template"}, defaultValue = "endpoints-template", description = "The template directory. Default is ${DEFAULT-VALUE}")
-	private Path templateDir = Paths.get(System.getProperty("user.dir"), "endpoints-templates");
+	private Path templateDir ;
 
 	@SuppressWarnings("CanBeFinal")
-    @Option(names = {"-o", "--output"}, description = "The directory to write the generated code to. Default is ~/endpoints-output")
+    @Option(names = {"-o", "--output"}, defaultValue = "endpoints-output", description = "The directory to write the generated code to. Default is ${DEFAULT-VALUE}")
-	private Path outputDir = Paths.get(System.getProperty("user.dir"), "endpoints-output");
+	private Path outputDir ;
 
 	@SuppressWarnings("unused")
     @Option(names = {"-v", "--verbose"}, description = "Print verbose debug messages.")

states-templates/Codecs.scala.ftl

@@ -0,0 +1,19 @@
+// Copyright 2025 "Johan Maasing" <johan@zoom.nu>
+//
+//   Licensed under the Apache License, Version 2.0 (the "License");
+//   you may not use this file except in compliance with the License.
+//   You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+//   Unless required by applicable law or agreed to in writing, software
+//   distributed under the License is distributed on an "AS IS" BASIS,
+//   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+//   See the License for the specific language governing permissions and
+//   limitations under the License.
+package ${config.package}
+
+object Codecs:
+<#list typeDefinitions as type>
+    given Codec[${type.name?cap_first}] = deriveCodec
+</#list>

states-templates/Types.scala.ftl

@@ -0,0 +1,23 @@
+// Copyright 2025 "Johan Maasing" <johan@zoom.nu>
+//
+//   Licensed under the Apache License, Version 2.0 (the "License");
+//   you may not use this file except in compliance with the License.
+//   You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+//   Unless required by applicable law or agreed to in writing, software
+//   distributed under the License is distributed on an "AS IS" BASIS,
+//   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+//   See the License for the specific language governing permissions and
+//   limitations under the License.
+package ${config.package}
+
+object StateAndMessageTypes:
+<#list typeDefinitions?sort as type>
+    case class ${type.name?cap_first}(
+    <#list type.fields as field>
+        ${field.name} : ${field.type},
+    </#list>
+    )
+</#list>

test01.states

@@ -1,4 +1,21 @@
-{ title: SomeNodes }
+/*
-start(s:S) -> middle: message(foo:bar),
+ Copyright 2025 "Johan Maasing" <johan@zoom.nu>
-middle -> middle: selfmessage,
+
-middle -> end: endmessage(bar:baz)
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+
+{ title: SomeNodes, package: nu.zoom.dsl.states }
+
+start(s:S) -> middle(foo:foo): message(foo:foo),
+middle -> middle(bar:bar): selfmessage(bar:bar),
+middle -> end: message(bar:baz)