Often when solving problems related to computer vision, the lack of data becomes a big problem. This is especially true when working with neural networks.
How great would it be if we had a limitless source of new original data?
This thought prompted me to develop a Domain Specific Language that allows you to create images in various configurations. These images can be used to train and test machine learning models. As the name suggests, generated DSL images can usually only be used in a narrowly focused area.
Language requirements
In my particular case, I need to focus on object detection. The language compiler must generate images that meet the following criteria:
- images contain different forms (for example, emoticons);
- the number and position of individual figures is customizable;
- image size and shapes are customizable.
The language itself should be as simple as possible. I want to determine the size of the output image first and then the size of the shapes. Then I want to express the actual configuration of the image. To keep things simple, I think of the image as a table, where each shape fits into a cell. Each new row is filled with forms from left to right.
Implementation
I chose a combination of ANTLR, Kotlin and Gradle to create the DSL . ANTLR is a parser generator. Kotlin is a JVM-like language similar to Scala. Gradle is a build system similar to
sbt.
Necessary environment
You will need Java 1.8 and Gradle 4.6 to complete the steps described.
Initial setup
Create a folder to contain the DSL.
> mkdir shaperdsl
> cd shaperdslCreate a file
build.gradle. This file is needed to list project dependencies and configure additional Gradle tasks. If you want to reuse this file, you only need to change the namespaces and the main class.
> touch build.gradleBelow is the content of the file:
buildscript {
ext.kotlin_version = '1.2.21'
ext.antlr_version = '4.7.1'
ext.slf4j_version = '1.7.25'
repositories {
mavenCentral()
maven {
name 'JFrog OSS snapshot repo'
url 'https://oss.jfrog.org/oss-snapshot-local/'
}
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.1'
}
}
apply plugin: 'kotlin'
apply plugin: 'java'
apply plugin: 'antlr'
apply plugin: 'com.github.johnrengelman.shadow'
repositories {
mavenLocal()
mavenCentral()
jcenter()
}
dependencies {
antlr "org.antlr:antlr4:$antlr_version"
compile "org.antlr:antlr4-runtime:$antlr_version"
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
compile "org.apache.commons:commons-io:1.3.2"
compile "org.slf4j:slf4j-api:$slf4j_version"
compile "org.slf4j:slf4j-simple:$slf4j_version"
compile "com.audienceproject:simple-arguments_2.12:1.0.1"
}
generateGrammarSource {
maxHeapSize = "64m"
arguments += ['-package', 'com.example.shaperdsl']
outputDirectory = new File("build/generated-src/antlr/main/com/example/shaperdsl".toString())
}
compileJava.dependsOn generateGrammarSource
jar {
manifest {
attributes "Main-Class": "com.example.shaperdsl.compiler.Shaper2Image"
}
from {
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
}
task customFatJar(type: Jar) {
manifest {
attributes 'Main-Class': 'com.example.shaperdsl.compiler.Shaper2Image'
}
baseName = 'shaperdsl'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}Language parser
The parser is built like ANTLR grammar .
mkdir -p src/main/antlr
touch src/main/antlr/ShaperDSL.g4with the following content:
grammar ShaperDSL;
shaper : 'img_dim:' img_dim ',shp_dim:' shp_dim '>>>' ( row ROW_SEP)* row '<<<' NEWLINE* EOF;
row : ( shape COL_SEP )* shape ;
shape : 'square' | 'circle' | 'triangle';
img_dim : NUM ;
shp_dim : NUM ;
NUM : [1-9]+ [0-9]* ;
ROW_SEP : '|' ;
COL_SEP : ',' ;
NEWLINE : '\r\n' | 'r' | '\n';Now you can see how the structure of the language becomes clearer. To generate the grammar source code, run:
> gradle generateGrammarSourceAs a result, you will get the generated code in
build/generate-src/antlr.
> ls build/generated-src/antlr/main/com/example/shaperdsl/
ShaperDSL.interp ShaperDSL.tokens ShaperDSLBaseListener.java ShaperDSLLexer.interp ShaperDSLLexer.java ShaperDSLLexer.tokens ShaperDSLListener.java ShaperDSLParser.javaAbstract syntax tree
The parser converts the source code into an object tree. The object tree is what the compiler uses as a data source. To get the AST, you first need to define the tree metamodel.
> mkdir -p src/main/kotlin/com/example/shaperdsl/ast
> touch src/main/kotlin/com/example/shaper/ast/MetaModel.ktMetaModel.ktcontains the definitions of the object classes used in the language, starting at the root. They all inherit from the Node . The tree hierarchy is visible in the class definition.
package com.example.shaperdsl.ast
interface Node
data class Shaper(val img_dim: Int, val shp_dim: Int, val rows: List<Row>): Node
data class Row(val shapes: List<Shape>): Node
data class Shape(val type: String): NodeNext, you need to match the class with ASD:
> touch src/main/kotlin/com/example/shaper/ast/Mapping.ktMapping.ktis used to build an AST using the classes defined in MetaModel.kt, using data from the parser.
package com.example.shaperdsl.ast
import com.example.shaperdsl.ShaperDSLParser
fun ShaperDSLParser.ShaperContext.toAst(): Shaper = Shaper(this.img_dim().text.toInt(), this.shp_dim().text.toInt(), this.row().map { it.toAst() })
fun ShaperDSLParser.RowContext.toAst(): Row = Row(this.shape().map { it.toAst() })
fun ShaperDSLParser.ShapeContext.toAst(): Shape = Shape(text)The code on our DSL:
img_dim:100,shp_dim:8>>>square,square|circle|triangle,circle,square<<<Will be converted to the following ASD:
Compiler
The compiler is the last part. He uses ASD to obtain a specific result, in this case, an image.
> mkdir -p src/main/kotlin/com/example/shaperdsl/compiler
> touch src/main/kotlin/com/example/shaper/compiler/Shaper2Image.ktThere is a lot of code in this file. I will try to clarify the main points.
ShaperParserFacadeIs a wrapper on top ShaperAntlrParserFacadethat builds the actual AST from the provided source code.
Shaper2Imageis the main compiler class. After it receives the AST from the parser, it goes through all the objects inside it and creates graphic objects, which it then inserts into the image. Then it returns the binary representation of the image. There is also a function mainin the companion object of the class to allow testing.
package com.example.shaperdsl.compiler
import com.audienceproject.util.cli.Arguments
import com.example.shaperdsl.ShaperDSLLexer
import com.example.shaperdsl.ShaperDSLParser
import com.example.shaperdsl.ast.Shaper
import com.example.shaperdsl.ast.toAst
import org.antlr.v4.runtime.CharStreams
import org.antlr.v4.runtime.CommonTokenStream
import org.antlr.v4.runtime.TokenStream
import java.awt.Color
import java.awt.image.BufferedImage
import java.io.ByteArrayInputStream
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.InputStream
import javax.imageio.ImageIO
object ShaperParserFacade {
fun parse(inputStream: InputStream) : Shaper {
val lexer = ShaperDSLLexer(CharStreams.fromStream(inputStream))
val parser = ShaperDSLParser(CommonTokenStream(lexer) as TokenStream)
val antlrParsingResult = parser.shaper()
return antlrParsingResult.toAst()
}
}
class Shaper2Image {
fun compile(input: InputStream): ByteArray {
val root = ShaperParserFacade.parse(input)
val img_dim = root.img_dim
val shp_dim = root.shp_dim
val bufferedImage = BufferedImage(img_dim, img_dim, BufferedImage.TYPE_INT_RGB)
val g2d = bufferedImage.createGraphics()
g2d.color = Color.white
g2d.fillRect(0, 0, img_dim, img_dim)
g2d.color = Color.black
var j = 0
root.rows.forEach{
var i = 0
it.shapes.forEach {
when(it.type) {
"square" -> {
g2d.fillRect(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
}
"circle" -> {
g2d.fillOval(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
}
"triangle" -> {
val x = intArrayOf(i * (shp_dim + 1), i * (shp_dim + 1) + shp_dim / 2, i * (shp_dim + 1) + shp_dim)
val y = intArrayOf(j * (shp_dim + 1) + shp_dim, j * (shp_dim + 1), j * (shp_dim + 1) + shp_dim)
g2d.fillPolygon(x, y, 3)
}
}
i++
}
j++
}
g2d.dispose()
val baos = ByteArrayOutputStream()
ImageIO.write(bufferedImage, "png", baos)
baos.flush()
val imageInByte = baos.toByteArray()
baos.close()
return imageInByte
}
companion object {
@JvmStatic
fun main(args: Array<String>) {
val arguments = Arguments(args)
val code = ByteArrayInputStream(arguments.arguments()["source-code"].get().get().toByteArray())
val res = Shaper2Image().compile(code)
val img = ImageIO.read(ByteArrayInputStream(res))
val outputfile = File(arguments.arguments()["out-filename"].get().get())
ImageIO.write(img, "png", outputfile)
}
}
}
Now that everything is ready, let's build the project and get a jar file with all dependencies ( uber jar ).
> gradle shadowJar
> ls build/libs
shaper-dsl-all.jarTesting
All we have to do is check if everything works, so try entering this code:
> java -cp build/libs/shaper-dsl-all.jar com.example.shaperdsl.compiler.Shaper2Image \
--source-code "img_dim:100,shp_dim:8>>>circle,square,square,triangle,triangle|triangle,circle|square,circle,triangle,square|circle,circle,circle|triangle<<<" \
--out-filename test.pngA file will be created:
.pngwhich will look like this:
Conclusion
It is a simple DSL, unsecured, and will likely break if misused. However, it suits my purpose well and I can use it to create any number of unique image samples. It can be easily extended for more flexibility and can be used as a template for other DSLs.
A complete DSL example can be found in my GitHub repository: github.com/cosmincatalin/shaper .