Chapter 12. Case Study-Meetup

We covered a practical scenario of intranet portal, which showcased powerful capabilities of the stack in Chapter 6, Elastic Stack in Action. The case was more of a log management rather than data analysis, and throughout the book, we have learned that Elastic Stack can do magic when it comes to data analytics. With the help of Kibana, you can bring numbers to a colorful presentation, which ultimately helps you to bring sense out of it to make impactful changes to your setup and drives you to take strategic decisions.

This chapter takes you to meetup(s) and helps you analyze a real, practical scenario. We will collect meetup data, find out the statistics, popular meetup, and so on. The series of topics we will cover in this chapter are as follows:

• Understanding meetup scenario

• Setting things up

• Analyzing data using Kibana

Understanding meetup scenario

The world might be moving toward nuclear families, but the increasing number of meetups everyday shows that the humanity still needs and considers interactions with each other. A meetup is a gathering in which people meet and do stuff, share knowledge, and plan things of their common interest. This is backed by a software platform developed and provided by Meetup Inc. launched in June 2002. This platform is available at https://www.meetup.com/ and open for everyone to join and do meetups. You can get the idea of meetups popularity by the fact that there are more than 29 million users, 260k+ groups across 180+ countries with monthly meetups count just more than 450k on an average.

These meetups are not limited to technologies, and people do gather for trekking, hiking, traveling, environment-related and meditation-related reasons, and other causes. All of this gives us a fantastic opportunity to do a good analysis over meetups happening all over the world. Similar...

Setting things up

Just like any other scenario, we need to set up Elasticsearch, Logstash or Beats or both, and Kibana for visualization. We would need something specific to capture data from meetups. Tricky thing is to decide whether we go for Logstash or Beats or a combination of both.

There was an input plugin developed for Logstash, which could read data for meetups, but that is not enough for us since we want to read more data and we need to go for another option. There is no beat available to capture the data from meetup as well.

The existing plugin is available at https://github.com/logstash-plugins/logstash-input-meetup/. Elastic site has also documented this plugin at https://www.elastic.co/guide/en/logstash/5.1/plugins-inputs-meetup.html. This plugin captures meetup

events using either

• venue_id: Multiple IDs can be provided, separated by commas

• group_id: Multiple IDs can be provided, separated by commas

• group_urlname: Path to group from meetup.com

The restriction this plugin brings...

Analyzing data using Kibana

When kibana is up and running, we can start visualizing meetup data. Before we can do that, we need to configure index for Kibana. Our index name in Elasticsearch is meetup as we specified in output configuration of Logstash. Let's set up our index pattern. Navigate to Kibana | Management | Index Patterns | Configure an Index Pattern or a Add New index. Add the index pattern for meetup and keep @timestamp field for time field name.

Note

We could use * as wildcard as well and keep meetup*, but we have only one index, and we know that it's meetup so no need to use wildcards. That will also help us to keep data from being mixed up in case we have other indices with the same pattern e.g. meetup-india, meetup-us etc.

Click on Create, and index pattern for our index would be set:

This would add the index pattern, and all fields present in the index meetup will be listed along with their types.

Filtering Content

As soon as our index pattern is set, discover page will be...

Getting Notified

Analyzing content has a purpose, mostly strategic. If this analysis was done by a company that deals in people relationship, they might be interested to know any new meetup so that they can connect to more people. If any meetup in city has good amount of RSVPs, there is a fair chance that 3-4 persons of the company's 'interest can be found. It usually happens during meetups. People of similar interests get to meet. What if, we can get a notification of some kind whenever a meetup is created. We can use X-Pack's alerting capabilities-watcher to mention specifically. The notification can be a log, e-mail, or some other kind supported by watcher. To learn more about watcher, refer to chapter 10, X-Pack: Alerting, Graph, and Reporting.

Let's create a watch for this use case. We would need a schedule, input, a condition, and an action, which will be used to notify us:

• Schedule: Assuming that one iteration takes place in one hour and data is updated, we will keep interval for our...

Summary

Meetups make a good case for analysis. In fact, we learned about meetups using a number of visualizations. We learned about why the existing plugin could not satisfy our needs and got to understand what new plugin offers. It is always interesting to find out facts of things we usually do. Meetup, to that matter, is a popular platform these days and helps to spread knowledge and good deeds. We learned how we could use tile maps to see the regions where most meetups are. Tag clouds helped us to find out the popular topics and categories under which a huge number of meetups take place. Finally, we saw a glimpse of how watcher could help us get notified of the events.

Elastic Stack offers more than what we could cover in the book. The tool set is still evolving and will be offering a lot. We hope the topics and use cases we chose to include in the book is a part of right set and helps you. We hope that the book helped you to learn how to effectively utilize Elastic Stack for your use...