16 posts tagged with "DynamoDB"

Last week, someone emailed me to ask about a potential cost optimization mechanism in DynamoDB. More on the specifics of that situation below, but the basic point is they were thinking about adding some additional application and architectural complexity because they were concerned about high DynamoDB costs for a particular use case.

I responded the way I always respond for these requests -- "have you done the math?"

One of my favorite things about DynamoDB is that you can easily do the math when considering how much it will cost you. I use this all the time in a few different ways, from getting a rough guess at how much DynamoDB will cost for my application to deciding between different approaches to solving a specific access pattern.

In 2007, a group of engineers from Amazon published The Dynamo Paper, which described an internal database used by Amazon to handle the enormous scale of its retail operation. This paper helped launch the NoSQL movement and led to the creation of NoSQL databases like Apache Cassandra, MongoDB, and, of course, AWS's own fully managed service, DynamoDB.

Fifteen years later, the folks at Amazon have released a new paper about DynamoDB. Most of the names have changed (except for AWS VP Swami Sivasubramanian, who appears on both!), but it's a fascinating look at how the core concepts from Dynamo were updated and altered to provide a fully managed, highly scalable, multi-tenant cloud database service.

In this post, I want to discuss my key takeaways from the new DynamoDB Paper.

One of the core complaints I hear about DynamoDB is that it can't be used for critical applications because it only provides eventual consistency.

It's true that eventual consistency can add complications to your application, but I've found these problems can be handled in most situations. Further, even your "strongly consistent" relational databases can result in issues if you're not careful about isolation or your choice of ORM. Finally, the benefits from accepting a bit of eventual consistency can be pretty big.

In this post, I want to dispel some of the fear around eventual consistency in DynamoDB.

I've written and spoken a lot about data modeling with DynamoDB over the years. I love DynamoDB, and I feel like I understand its pros and cons pretty well.

Recently, the topic of the compatibility of GraphQL and single-table design in DynamoDB came up again, sparked by a tweet from Rick Houlihan that led to a follow-up tweet indicating that I should update my post about single-table design where I mentioned that GraphQL might be an area where you don't want to use single-table design with DynamoDB.

Twitter is a bad medium for nuance, and I think the question of whether to use single-table design with GraphQL is a nuanced one that depends a lot on why you are choosing to use GraphQL.

Prefer video? View this post on YouTube!

DynamoDB powers some of the highest-traffic systems in the world, including Amazon.com's shopping cart, real-time bidding for ad platforms, and low-latency gaming applications. They use DynamoDB because of its fast, consistent performance at any scale.

Before I understood DynamoDB, I thought AWS had a giant supercomputer that was faster than everything else out there. Turns out that's not true. They're not defying the laws of physics -- they're using basic computer science principles to provide the consistent, predictable scaling properties of DynamoDB.

In this post, we'll take a deep look at DynamoDB partitions -- what they are, why they matter, and how they should affect your data modeling. The most important reason to learn about DynamoDB partitions is because it will shape your understanding of why DynamoDB acts as it does. At first glance, the DynamoDB API feels unnecessarily restrictive and the principles of single-table design seem bizarre. Once you understand DynamoDB partitions, you'll see why these things are necessary.

Prefer video? View this post on YouTube!

As a programmer, it is important to know the limits of any service that you're using. In some cases, the limits of a particular service may make it unsuitable for the task at hand, such as using Route53 as a database. In other cases, the limit may alter how you structure your solution, such as how the 15 minute limit on Lambda execution time requires you to break down large work into smaller chunks.

In this post, we'll talk about limits in DynamoDB. While the DynamoDB documentation has a long list of limits for the service, most of these are not going to drastically change how you use DynamoDB. For example, nested attributes can only go to 32 levels of depth, I've never found this to be a factor in building my applications.

If you're working with DynamoDB, you're likely to rely on Condition Expressions when manipulating items in your table. Condition Expressions can ensure you don't overwrite existing users, allow bank account balances to drop below \$0, or give Admin access to every user in your application.

Yet despite their usefulness, I see Condition Expressions misunderstood quite often. My hunch is that this is due to an underdeveloped mental model of how DynamoDB works and why it makes the choices it makes.

In this post, you'll learn all about DynamoDB's Condition Expressions. First, we'll start with some background on what Condition Expressions are. We'll see why they are helpful and the API operations to which they apply.

DynamoDB is sometimes considered just a simple key-value store, but nothing could be further from the truth. DynamoDB can handle complex access patterns, from highly-relational data models to time series data or even geospatial data.

In this post, we'll see how to model one-to-many relationships in DynamoDB. One-to-many relationships are at the core of nearly all applications. In DynamoDB, you have a few different options for representing one-to-many relationships.

This is part 2 of a two-part post on DynamoDB Transactions. Check out part 1 in this series for a look at how and when to use DynamoDB Transactions.

In this post, we're going to do some performance testing of DynamoDB Transactions as compared to other DynamoDB API calls. As a reminder from the last post, you can use DynamoDB Transactions to make multiple requests in a single call. Your entire request will succeed or fail together -- if a single write cannot be satisfied, all other writes will be rolled back as well.

Amazon's DynamoDB was released in 2012 and has been adding a drumbeat of new features ever since. It's hard to believe now, but the original version of DynamoDB didn't have DynamoDB Streams, parallel scans, or even secondary indexes.

One of the more exciting feature releases from DynamoDB in recent years has been the addition of DynamoDB Transactions at re:Invent 2018. With DynamoDB Transactions, you can write or read a batch of items from DynamoDB, and the entire request will succeed or fail together.

This feature release simplified a lot of workflows that involved complex versioning and multiple requests to accurately work across multiple items. In this post, we'll review how and why to use DynamoDB Transactions.