Introduction
Elasticsearch, the heart of the Elastic Stack, is a search and analytics engine that can handle all types of data, such as textual, numerical, geospatial, structured, and unstructured. It stores data as JSON documents in an index, which is a collection of related documents1. It is fast, scalable, and distributed.
Some other components of the Elastic Stack are:
- Kibana: a visualization and management tool that lets you explore and interact with your data in Elasticsearch.
- Beats: a family of lightweight agents that collect data from different sources and send it to Elasticsearch or Logstash.
- Logstash: a data processing pipeline that can ingest data from various sources, transform it, and output it to different destinations.
Pros and Cons:
Pros:
- It can handle large amounts of data and scale horizontally by adding more nodes to the cluster.
- It can handle a variety of data types and formats, including structured, semi-structured and unstructured data.
- It has a strong team of developers and product managers working on improving it constantly.
- It has a simple REST API that makes it easy to interact with.
- It has a rich ecosystem of tools and plugins that extend its functionality.
Cons:
- It can be complex to set up and maintain, especially for beginners or small teams.
- It can be prone to instability if the indices exceed the data storage limitations of the nodes or if there are network issues.
- It can be expensive to run on cloud platforms or on-premise hardware, depending on the size and volume of data.
- It can have security issues if not configured properly or if using older versions.
We are currently not using any parts of the Elastic Stack at task.vc in any of our products.
We have tried in the past, but found the stack to be really heavy on ressource usage compared to clickhouse. Also configuring a cluster and indexes correctly to the individual usecase can be quite challenging from our experience, even when using Elastic Cloud.
Regards
Phil from task.vc
