The source code for this article can be found on Github.
Using Redis as a best effort locking mechanism can be very useful in practice, to prevent two logical threads from clobbering each other. While redis locking is certainly not perfect, and you shouldn’t use redis locking if the underlying operation can’t be occasionally done twice, it can still be useful for that “best effort, do this once” use case.
The source code for what follows can be found on github.
In a previous article, we showed how to efficiently execute a lua script in redis using lettuce. To really scale our caching solution horizontally [and elegantly deal with many scaling headaches], we will also want to make sure we can execute our lua scripts against clustered redis, which as we’ll see here is pretty straightforward.
As of this writing, there are a solid twenty or so commands you can execute against redis for the list data type. This article will be walking through some of the more common operations you are likely to need when interacting with redis and lists using lettuce, and the source code can be found on Github.
In clustered redis, any non hash tagged key can be sent unpredictably [well, actually predictably, if you know the formula] to any given primary node in the cluster. The very basic way it works is:
The source code for this article can be found on Github.
Lettuce is a redis client with reactive support. There is a super handy embedded redis for java project out there, and this kind of integration testing inside your service is worth its weight in gold, in my humble opinion. This post will detail how to merge both of these worlds together, and set up redis integration tests when you’re using a lettuce client.
Nested attributes in DynamoDB are a way to group data within an item together. The attributes are said to be nested if they are embedded within another attribute.
DynamoDB’s Local Secondary Indexes allow for more query flexibility than a traditional partition and range key combination. They are also the only index in DynamoDB where a strongly consistent read can be requested [global secondary indexes, the other index that dynamo supports, can at best be eventually consistent]. I will walk through an example for how to use local secondary indexes in dynamo using the AWS SDK 2.0 for Java, which has full reactive support, in this post.
In this post, we’ll demonstrate how expiring items in DynamoDB works in java, using the AWS SDK 2.0+, which has full reactive support.
Queries in DynamoDB allow you to find data. This is only an option to you if your table has a partition and sort key.
Amazon SNS is AWS’s solution to pub/sub. In a large, distributed system, decoupling events from services that need to act on those events allows for teams that own different services to better work in parallel, and also prevents the need for coordinating code deploys to deliver new features. If a services is already publishing a generic event, other services can hook into that event and act on them without needing anything but a bit of infrastructure.
DynamoDB Streams is AWS’s home grown Change Data Capture [CDC] mechanism, which allows the consumer of the stream to see records probably in approximately the order they were created [it’s basically impossible, at scale, to guarantee that all records across all partitions will somehow stream the data in exactly the same order that it was written]. This is a pretty fantastic feature because it allows us to reliably do —something— after we add new data, update existing data, or delete existing data. As long as all the stream records are read and processed, we can ensure at least once processing on data changes and then go sleep soundly at night knowing that there is one less edge case in our application. Combine that with the natural scale that DynamoDB provides via its leaderless architecture and you can build this thing once and probably never have to worry about it handling more load ever again.
DynamoDB is a fully managed distributed datastore that does a great job of alleviating the operational overhead of building very scalable systems.
