I'm trying to find a solution for dealing with a data set of logs of about 500Gb. Ideally what I search for is an index over log message keywords that I parse from the log messages. Like:
Log message 2019-05-06 12:43:32.133 ERROR [ 23][Repository] Updating record 356/23:2,4 Keywords: repository, updating, record, "356/23:2,4"
That search over the keywords like
select from logs where keywords contains '356/23:2,4' (not an SQL).
So far I've tried DBMS proof-of-concept with keywords table, logs table and keywords-logs relation table. It works but to put it mildly, it is kind of not ideal. I've also tried influxdb, but it does not support multiple values per tag (otherwise it would be exactly what I looking for).
It seems to be quite basic stuff, so I hope there is just a right solution for that, ideally installed locally (not in cloud) and opensource. Any recommendation?