zoom/endgen
1
0
Fork 0
Code Issues Pull requests Projects Releases 4 Packages Wiki Activity Actions

Documentation and merge behaviour for state files

Browse source
This commit is contained in:
Johan Maasing 2025-04-19 19:24:45 +02:00
parent a546d257f3
commit cf3ff3f982
6 changed files with 193 additions and 32 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

104
README.md
View file

@ -32,6 +32,17 @@ parser and a code generator using [freemarker](https://freemarker.apache.org).
| mytemplate.xxx.ftl |
\____________________\
```
Endgen currently contains two separate parsers:
* endpoint - A DSL for expressing HTTP endpoints.
* state - A DSL for expressing state and transitions.
Which parser that is used to read the input file is determined by the file name ending
'.endpoints' or '.states' or by a command line argument.
The endpoint DSL and the state DSL share the grammar for expressing configuration and data types
,see below for details.
## How to Run
You need a Java 21 (or later) runtime and java in the path. A very convenient way to install a java runtime is [SdkMan](https://sdkman.io).
@ -39,20 +50,23 @@ Unpack the archive, run the provided shellscript file.
### Usage
```
Usage: run.sh [-hvV] [-o=<outputDir>] [-t=<templateDir>] <file>
sage: run.sh [-hvV] [-o=<outputDir>] [-p=<parser>] [-t=<templateDir>] <file>
Generate source code from an endpoints specification file.
<file> The source endpoints DSL file.
-h, --help Show this help message and exit.
-o, --output=<outputDir> The directory to write the generated code to.
Default is ~/endpoints-output
Default is endpoints-output
-p, --parser=<parser> Force use of a specific parser instead of
determining from filename. Valid values:
Endpoints, States.
-t, --template=<templateDir>
The template directory. Default is
~/endpoints-templates
endpoints-template
-v, --verbose Print verbose debug messages.
-V, --version Print version information and exit.
```
## DSL example
## Endpoint DSL example
In the simplest form the DSL looks like this
```
/some/endpoint <- SomeType(foo:String)
@ -80,6 +94,8 @@ This is the ANTLR grammar for the root of the DSL
```antlrv4
document : generatorconfig? (namedTypeDeclaration|endpoint)* ;
```
### Configuration block
Meaning that the DSL file has an optional `generatorconfig` block at the top. Then you can write either; a type
definition, or an endpoint declaration, as many times as you like.
@ -99,7 +115,9 @@ Embedded(foo:Bar)
This consists of a config block with 2 items, the 'package' and the 'mykey' definition. These are available to be used
in the freemarker template as a Map of String-keys to String-values.
`/some/endpoint <- SomeType(foo:String)` is an endpoint declaration. It declares one endpoint that have a request body
### Endpoint definition
`/some/endpoint <- SomeType(foo:String)` is an endpoint definition. It declares one endpoint that have a request body
data type called `SomeType` that has a field called `foo` of the type `String`.
### Data types
@ -110,7 +128,7 @@ just named field-name and the other string is named field-type.
`Embedded(foo:Bar)` is a `namedTypeDeclaration` which is parsed the same way as the request type above. But isn't tied
to a specific endpoint.
### Automatically named data types
### Automatically named endpoint data types
`/some/other/endpoint <- (bar:Seq[Embedded])` is another endpoint declaration. However this time the request body is
not named in the DSL. But all datatypes must have a name so it will simply name it after the last path segment and
tack on the string 'Request' at the end. So the AST till contain a datatype named `endpointRequest` with a field named
@ -122,7 +140,7 @@ decide to generate in the templates.
The only 'semantic' validation the parser performs is to check that not two types have the same name.
### Reponse data types
### Endpoint reponse data type
It is possible to have an optional response data type declared like so:
`/some/other/endpoint <- (bar:Seq[Embedded]) -> ResponseType(foo: Bar)`
@ -130,9 +148,45 @@ It is possible to have an optional response data type declared like so:
The right pointing arrow `->` denotes a response type, it can be an anonymous data type in which case the parser till
name it from the last path segment and add 'Response' to the end of the data type name.
### DSL config
The only key in the config block the generator looks at is called `ending`, this will be used as the file ending for
the resulting file of applying the freemarker template.
### State grammar
This is an example of a state file:
```
start -> middle: message,
middle -> middle: selfmessage,
middle -> end: endmessage
```
It contains 3 state definitions `start`, `middle` and `end`. A state definition will be parsed as a data type with
the name of the state as the type name.
It also contains 3 message definitions `message`, `selfmessage` and `endmessage`. Message definitions will also be
parsed as data types.
Since the parser will extract datatypes it is possible to define the fields of the data types. This is a slightly more
complicated example:
```
start(foo:Foo) -> middle: message(a: String),
middle(bar:Bar) -> middle: selfmessage,
middle -> end: endmessage
```
Where for example the data type for `middle` will have the field declaration with the name `bar` and the type `Bar`.
Fields for the same state data type, or message data type, will be merged. Here is a complex example:
```
start(s:S) -> middle(foo:foo): message(foo:foo),
middle -> middle(bar:bar): selfmessage(bar:bar),
middle -> end: message(bar:baz)
```
Not that we can declare fields on both the `from` and `to` state declarations. The `middle` datat type will have field
definitons for `foo` and `bar`.
The data type for `message` will have fields for `foo` and `bar`.
One restriction is that states and messages may not have the same name, i.e. be parsed as the same data type.
## Generating
If the parser is successful it will hold the following data in the AST
@ -140,12 +194,17 @@ If the parser is successful it will hold the following data in the AST
```java
public record DocumentNode(
Map<String, String> config,
List<TypeNode> typeDefinitions,
List<EndpointNode> endpoints) {
Set<TypeNode> typeDefinitions,
List<EndpointNode> endpoints,
Set<StateNode> states) {
}
```
This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your freemarker template like this:
Depending on the parser used the endpoints or the states will be null but config and typeDefinitions are populated the
same for both parsers.
This will be passed to the freemarker engine as the 'root' data object, meaning you have access to the parts in your
freemarker template like this:
```injectedfreemarker
<#list typeDefinitions as type>
@ -153,7 +212,7 @@ This will be passed to the freemarker engine as the 'root' data object, meaning
</#list>
```
That is, you can directly reference `typeDefinitions`, `endpoints` or `config`.
That is, you can directly reference `typeDefinitions`, `endpoints`, `states` or `config`.
### Config
The config object is simply a String-map with the keys and values unfiltered from the input file. Here is an example
@ -213,3 +272,20 @@ Output data type.
</#list>
```
### States
The set of states will hold items of this shape:
```injectedfreemarker
public record StateNode(String name, String data, Set<TransitionNode> transitions) {
}
```
and the transitions has this structure:
```injectedfreemarker
public record TransitionNode(String message, String toState) {
}
```

44
parser/src/main/java/nu/zoom/dsl/ast/StatesVisitorTransformer.java
View file

@ -62,23 +62,49 @@ public class StatesVisitorTransformer extends StatesBaseVisitor<StatesParser.Doc
}
public Set<TypeNode> getTypes() {
// TODO calculate data types from NodeTypes and MessageTypes with duplicate check.
HashMap<String, TypeNode> typeNodes = new HashMap<>();
final HashMap<String, TypeNode> stateTypeNodes = new HashMap<>();
this.nodeTypes.forEach(typeNode -> {
if (typeNodes.containsKey(typeNode.name())) {
throw new RuntimeException("Duplicate type name: " + typeNode.name());
if (stateTypeNodes.containsKey(typeNode.name())) {
TypeNode mergedNode = mergeTypeFields(typeNode, stateTypeNodes.get(typeNode.name()));
stateTypeNodes.put(typeNode.name(), mergedNode);
} else {
typeNodes.put(typeNode.name(), typeNode);
stateTypeNodes.put(typeNode.name(), typeNode);
}
}) ;
final HashMap<String, TypeNode> messageTypeNodes = new HashMap<>();
this.messageTypes.forEach(typeNode -> {
if (typeNodes.containsKey(typeNode.name())) {
throw new RuntimeException("Duplicate type name: " + typeNode.name());
if (stateTypeNodes.containsKey(typeNode.name())) {
throw new ParseException("Message " + typeNode.name() + " conflicts with state with the same type name");
}
if (messageTypeNodes.containsKey(typeNode.name())) {
TypeNode mergedNode = mergeTypeFields(typeNode, messageTypeNodes.get(typeNode.name()));
messageTypeNodes.put(typeNode.name(), mergedNode);
} else {
typeNodes.put(typeNode.name(), typeNode);
messageTypeNodes.put(typeNode.name(), typeNode);
}
}) ;
return Set.of(typeNodes.values().toArray(new TypeNode[0])) ;
HashSet<TypeNode> allTypeNodes = new HashSet<>(stateTypeNodes.values());
allTypeNodes.addAll(messageTypeNodes.values());
return allTypeNodes ;
}
private TypeNode mergeTypeFields(TypeNode t1, TypeNode t2) {
List<FieldNode> t1Fields = (t1 != null) ? t1.fields() : List.of() ;
List<FieldNode> t2Fields = (t2 != null) ? t2.fields() : List.of() ;
HashMap<String, FieldNode> mergedFields = new HashMap<>();
t1Fields.forEach(field -> {
if (mergedFields.containsKey(field.name())) {
throw new ParseException("Duplicate field name: " + field.name());
}
mergedFields.put(field.name(), field);
});
t2Fields.forEach(field -> {
if (mergedFields.containsKey(field.name())) {
throw new ParseException("Duplicate field name: " + field.name());
}
mergedFields.put(field.name(), field);
});
return new TypeNode(t1.name(), mergedFields.values().stream().toList()) ;
}
private List<FieldNode> extractFields(StatesParser.TypeDeclarationContext declaration) {

8
parser/src/main/java/nu/zoom/dsl/cli/EndpointsCLI.java
View file

@ -43,12 +43,12 @@ public class EndpointsCLI implements Callable<Integer> {
private Path file;
@SuppressWarnings("CanBeFinal")
@Option(names = {"-t", "--template"}, description = "The template directory. Default is ~/endpoints-templates")
private Path templateDir = Paths.get(System.getProperty("user.dir"), "endpoints-templates");
@Option(names = {"-t", "--template"}, defaultValue = "endpoints-template", description = "The template directory. Default is ${DEFAULT-VALUE}")
private Path templateDir ;
@SuppressWarnings("CanBeFinal")
@Option(names = {"-o", "--output"}, description = "The directory to write the generated code to. Default is ~/endpoints-output")
private Path outputDir = Paths.get(System.getProperty("user.dir"), "endpoints-output");
@Option(names = {"-o", "--output"}, defaultValue = "endpoints-output", description = "The directory to write the generated code to. Default is ${DEFAULT-VALUE}")
private Path outputDir ;
@SuppressWarnings("unused")
@Option(names = {"-v", "--verbose"}, description = "Print verbose debug messages.")

19
states-templates/Codecs.scala.ftl Normal file
View file

@ -0,0 +1,19 @@
// Copyright 2025 "Johan Maasing" <johan@zoom.nu>
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package ${config.package}
object Codecs:
<#list typeDefinitions as type>
given Codec[${type.name?cap_first}] = deriveCodec
</#list>

23
states-templates/Types.scala.ftl Normal file
View file

@ -0,0 +1,23 @@
// Copyright 2025 "Johan Maasing" <johan@zoom.nu>
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package ${config.package}
object StateAndMessageTypes:
<#list typeDefinitions?sort as type>
case class ${type.name?cap_first}(
<#list type.fields as field>
${field.name} : ${field.type},
</#list>
)
</#list>

25
test01.states
View file

@ -1,4 +1,21 @@
{ title: SomeNodes }
start(s:S) -> middle: message(foo:bar),
middle -> middle: selfmessage,
middle -> end: endmessage(bar:baz)
/*
Copyright 2025 "Johan Maasing" <johan@zoom.nu>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
{ title: SomeNodes, package: nu.zoom.dsl.states }
start(s:S) -> middle(foo:foo): message(foo:foo),
middle -> middle(bar:bar): selfmessage(bar:bar),
middle -> end: message(bar:baz)