IBM just announced Event Store, a hybrid datastore to store events. The originality? Events can be streamed in and it is based on Apache Spark. IBM claims to be able to store 1m events per second per node.
As it is based on Spark, it easily integrates with Java, Scala, Kafka, JDBC (Java DataBase Connectivity), IBM Streams, and more on the development/integration front. It saves its data in Parquet in S3, HDFS (Hadoop Distributed File System), or object storage like Cleversafe (now an IBM company), GlusterFS (hopefully my friends at Scality are in the loop as well).
Under the Hood
Its design allows ingestion and analytics at the same time. IBM Event Store is also thought with HA (High-Availability) in mind, so an engine can disappear, but the cluster stays. This is very certainly achieved by leveraging Spark’s cluster functionalities.
Watch Event Store in Action
IBM just released a video on YouTube, in less than 4 minutes, get a good sense of what it does. Thanks Phil for sharing.