Reader small image

You're reading from  Mastering Apache Storm

Product typeBook
Published inAug 2017
Reading LevelExpert
Publisher
ISBN-139781787125636
Edition1st Edition
Languages
Right arrow
Author (1)
Ankit Jain
Ankit Jain
author image
Ankit Jain

Ankit Jain holds a bachelor's degree in computer science and engineering. He has 6 years, experience in designing and architecting solutions for the big data domain and has been involved with several complex engagements. His technical strengths include Hadoop, Storm, S4, HBase, Hive, Sqoop, Flume, Elasticsearch, machine learning, Kafka, Spring, Java, and J2EE. He also shares his thoughts on his personal blog. You can follow him on Twitter at @mynameisanky. He spends most of his time reading books and playing with different technologies. When not at work, he spends time with his family and friends watching movies and playing games.
Read more about Ankit Jain

Right arrow

Trident state


Trident provides an abstraction for reading from and writing results to stateful sources. We can maintain the state either internally to the topology (memory), or we can store it in external sources (Memcached or Cassandra).

Let's consider that we are maintaining the output of the preceding hello world Trident topology in a database. Every time you process the tuple, the count of country present in a tuple is increased in the database. We can't achieve exactly-once processing by only maintaining a count in the database. The reason is that if any tuple failed during processing, then the failed tuple is retried. This gives us a problem while updating the state, because we are not sure whether the state of this tuple was updated previously or not. If the tuple has failed before updating the state, then retrying the tuple will increase the count in the database and make the state consistent. But if the tuple has failed after updating the state, then retrying the same tuple will...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Mastering Apache Storm
Published in: Aug 2017Publisher: ISBN-13: 9781787125636

Author (1)

author image
Ankit Jain

Ankit Jain holds a bachelor's degree in computer science and engineering. He has 6 years, experience in designing and architecting solutions for the big data domain and has been involved with several complex engagements. His technical strengths include Hadoop, Storm, S4, HBase, Hive, Sqoop, Flume, Elasticsearch, machine learning, Kafka, Spring, Java, and J2EE. He also shares his thoughts on his personal blog. You can follow him on Twitter at @mynameisanky. He spends most of his time reading books and playing with different technologies. When not at work, he spends time with his family and friends watching movies and playing games.
Read more about Ankit Jain