I’m currently looking at an EFK stack to aggregate log entries from multiple appliances in one spot. I’m curious how other people are solving this as I’m running into an issue with the timestamps of log entries.
[----] I, [2016-08-09T17:37:02.794408 #16835:1045988]
The log timestamp is affected by the appliance’s Timezone. It seems that Elasticsearch really wants UTC timestamps (otherwise I end up with Local time -4:00 being interpreted as UTC and Kibana does the conversion as well resulting in a log timestamp of 13:37:02.794408 for the example above).
I see 3 solutions to this:
Change the appliance timezone to UTC. This has the unfortunate side effect of things like the dashboard showing last run/next run in the future (with no TZ indication)
Have the appliance start appending the TZ information to log entries
Use fluentd to somehow mangle the time to include the TZ information and/or convert to UTC. I’m not sure this is possible as the time field is treated somewhat specially.
Any thoughts or suggestions from other folks who have solved this problem already?