Add KVIN ingestion benchmarks for CSV and JSON processing#52
Open
wmehling wants to merge 1 commit into
Open
Conversation
- Implement KvinIngestionCsvDiagnosticBenchmark for performance testing of CSV ingestion. - Create KvinIngestionJsonDiagnosticBenchmark for benchmarking JSON ingestion. - Introduce KvinIngestionWorkload to generate test data for benchmarks. - Add unit tests for KvinIngestionCsvDiagnosticBenchmark and KvinIngestionWorkload. - Remove obsolete KvinServiceBenchmark Scala file. - Document benchmark setup and execution in markdown format.
kenwenzel
reviewed
Jul 19, 2026
kenwenzel
left a comment
kenwenzel
left a comment
Member
There was a problem hiding this comment.
I propose to remove the use of reflection and to think about reworking the generation of tuples with more diversity in item, property and time.
| jsonPayload = workload.jsonPayload(); | ||
| csvPayload = workload.csvPayload(); | ||
| csvPayloads = workload.csvPayloads(SEQUENTIAL_CSV_FILE_COUNT); | ||
| try { |
Member
There was a problem hiding this comment.
AI often uses reflection to load classes from other modules - this is usually wrong.
Use something like
// create configuration and a model set factory
val module: KommaModule = ModelPlugin.createModelSetModule(classOf[ModelPlugin].getClassLoader)
val factory: IModelSetFactory = Guice.createInjector(new ModelSetModule(module)).getInstance(classOf[IModelSetFactory])
// create a model set with an in-memory repository
modelSet = factory.createModelSet(MODELS.NAMESPACE_URI.appendFragment("MemoryModelSet"))
Globals.contextModelSet.default.set(Full(modelSet))
``
as it was also contained in the deleted KvinServiceBenchmark.| workload = new KvinIngestionWorkload(); | ||
| csvPayload = workload.csvPayload(); | ||
| try { | ||
| KommaModule module = ModelPlugin.createModelSetModule( |
Member
There was a problem hiding this comment.
Replace by version without reflection.
| public static final int SEQUENCES_PER_TIMESTAMP = 5; | ||
| public static final long START_TIME = 1_710_000_000_000L; | ||
|
|
||
| public static final URI PROPERTY = URIs.createURI("http://iwu.lf.de/ecc4p/values"); |
Member
There was a problem hiding this comment.
Maybe it would be more realistic to slightly randomize this. Something like choosing 6 channels out of 100 randomly and one property out of 10.
| private final byte[] csvPayload; | ||
|
|
||
| public KvinIngestionWorkload() { | ||
| this.tuples = createTuples(); |
Member
There was a problem hiding this comment.
I don't fully understand the logic but it seems that only a fixed set of tuples with static timestamps is generated once and then inserted on each iteration?!
| } | ||
|
|
||
| private static JValue parseJson(String json) throws Exception { | ||
| Class<?> parserClass = Class.forName("org.json4s.native.JsonParser$"); |
Member
There was a problem hiding this comment.
use of reflection instead of direct class names
|
|
||
| @SuppressWarnings("unchecked") | ||
| private static Box<List<KvinTuple>> parseProductionJson(JValue json) throws Exception { | ||
| Class<?> parserClass = Class.forName("io.github.linkedfactory.service.util.JsonFormatParser$"); |
Member
There was a problem hiding this comment.
use of reflection instead of direct class names
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
How to review
Look at doc kvin-ingestion.md and run benchmarks and see results.
Results from my tests (max values)
Overall
CSV (as known) significantly faster with some headroom to faster parse so we can achieve closer to raw speed of DB around 900 k tuples/s
CSV diagnostics
Steps building on each other with more and more of the actual postCSV pipeline:
So would try to make the parsing and consuming faster in a seperate issue building on that branch.