transentis consulting

Implementation of a Large Scale Monitoring System

Imagine you are setting up a backend system that is providing services to over one million autonomous devices that are in constant, real-time communication with the backend – Internet of Things at scale.
Data from the devices is passed through a chain of systems and even routed to 3rd party systems for further computation. Results are then passed back to the devices in near real-time so that they can make autonomous decisions.

The Challenge

The challenge is to keep an overview of what is going in in this huge, distributed system. There is no lack of data about the backend system and the devices connected to it: on the contrary, you need to analyze huge amounts of it, present aggregated facts to decision makers and even make autonomous control decisions, all in near real-time.

The Solution: Elasticsearch to The Rescue

We designed and implemented a monitoring system based on the Elastic stack: using Logstash to filter and extract the relevant data, Elasticsearch to store it and provide fast search capabilities and Kibana to display it.
The system is running worldwide and consists of dozens of Elasticsearch clusters, the largest of which has over 100 nodes. Over 8 terabytes of data are ingested daily.

Our Expertise

  • Data strategies and architectures

Target Group

Upcoming Events

Training AI to Play The Beer Distribution Game

2 hours - 2021-10-27

Mastering The Complexity of Digital Transformation

2 hours - 2021-10-28

Modeling Growth Strategies For Professional Service Firms

2 hours - 2021-11-18

Go to event overview

Recent Blogposts

Meetup: Agile Transformation Tools

Olga Medvedeva - 2021-09-27

Meetup: Understanding The Beer Distribution Game

Olga Medvedeva - 2021-09-01

Introduction to The Business Prototyping Toolkit

Oliver Grasl - 2021-08-17

Go to blog overview

Email newsletter

Subscribe to our newsletter and stay up to date about the latest trends and news.

Name
Email
Subscribe to our newsletter and stay up to date about the latest trends and news.
  • transentis
Contact us

transentis consulting

Geisbergstraße 9

10777 Berlin


+49 30 9203833320

info@transentis.com