Open Distro for Elasticsearch on AWS

Intro

It's all started a year ago as AWS Team anounced Open Distro for Elasticsearch to the public. In this article I'll share my observations to Open Distro Status with the goal to keep overview for myself and for you, dear visitor.

Open Distro Package

First of all good news - it's not a fork. The Open Distro people do not want to increase fragmentation. Quote from FAQ

Open Distro for Elasticsearch is not a fork. In fact, we intend to continue making contributions to the Apache 2.0-licensed Elasticsearch code

and they want to be pretty in sync

Yes. We will regularly update Open Distro for Elasticsearch to remain current with the latest versions of the Apache 2.0 licensed open source versions of Elasticsearch and Kibana

and of course everything they contribute will be Apache 2.0 licensed.

Open Distro Features

Open Distro includes

  • Elasticsearch - no surprize
  • Kibana - no surprize
  • Security - Authentication and access control for the cluster
  • Alerting - Alerting Notifications based on data stream
  • SQL - SQL like query of your data
  • Index State Management - Automate index operations (like delete old)
  • Performance Analyzer - Monitor and optimize your cluster
  • Anomaly Detection - beta now

Three of them i've pointed out. Firts Open Distro comes with it's own Security Module that replaces proprietary licenced X-Pack from Elastic. Well actually this move for Open Distro has put Elastic into Zugzwang and they reacted with the opening of X-Pack how ever it's still not free and not Open Source lincenced.

Next feature is Alertings. Alertings give us a possibility to create Data Stream based Monitoring. This is nice in combination with Metricbeat which i plan to cover some day in another blogpost.

Last one i've wouild like to mentione here is the basic Index state Management. It's included and is usable out of Kibana, so no additional tools are no need anymore for this important task.

Open Distro Versions

Check Version History as we ca see releases are relatively up to date:

  • Open Distro 1.6.0 Release on 02 Apr 2020 - Elasticsearch version 7.6.1
  • Open Distro 1.4.0 Release on 10 Feb 2020 - Elasticsearch version 7.4.2

at least i can definitely live with that speed.

AWS Elasticsearch

Now we are running AWS Elasticsearch for a couple of month. AWS Managed Open Distro has some pros an cons in comparison to self manged installation.

Pros

  • AWS is dealing with configuration, Updates and other Operations of the cluster. That saves our Ops resources by that pretty much. Check what i mean

Cons

  • AWS has limited set of Features compared to Open Distro. What i saw before was a limited set on Index mamangent possbilities
  • It takes some time before latest Open Distro release becomes available.
  • Instances cost more that own VMs or Hardware
  • Currently it hard to get documentation or troubleshooting for cross ELK and OpenDistro cases :(

What are your experiences with Open Distro or your Opinion to Open Distro at AWS? Feel Free to share in the comments if you like.