ScyllaDB is a speciality database for high-throughput workloads that require ultra-low latency. The company has this month detailed a new release of ScyllaDB X Cloud, a service described as a “truly elastic database” that helps teams achieve predictable performance – all without the looming spectre of overprovisioning. But how good is this new technology and why do we over-order on the database menu in the first place?
With so many data instances, estates and repositories deployed in a format that leads to an inevitable level of overprovisioning, organisations are clearly looking for a route to more efficient data scaling management, but why does this overprovisioning happen?
Why does overprovisioning happen?
We know that overprovisioning is often endemic when teams are forced to align for more capacity than necessary as a risk aversion tactic to support peak load provisioning. Nobody wants databases to experience downtime and this is why it happens.
It also happens as a result of bundled resource constraints when cloud providers package up resources (CPU, RAM, analytics, AI calls and I/O) into fixed instance types. It also happens as a result of lack of real-time visibility into system requirements on a day-to-day (or even hour-by-hour) basis… and overprovisioning also comes about due to low quality query and schema optimisation (the database may feel slow due to inefficient SQL queries, missing indexes or poor schema design) and overprovisioning can occur due to inflexible legacy architectures that lack native elasticity in their DNA and are anchored by storage limitations (some cloud providers only allow volumes to be resized every six hours… and sometimes that’s a long time n data), so there is a lot to contend with in this space.
The new release of ScyllaDB X Cloud hopes to counter some (or all) of those issues with its new services, which include auto scaling, so that capacity can be increased rapidly and precisely when required, then scaled back as soon as the surge subsides.
Traditional scaling lag is a drag
For example, it can scale from 100K OPS to 2M operations (OPS) in minutes, with consistent single-digit millisecond P99 latency during scale, backup and failure recovery. As a result, teams don’t need to wastefully overprepare for potential peaks or suffer performance degradation due to traditional scaling lag.
The new release extends ScyllaDB’s “tablet-based elasticity” to use cases that involve ScyllaDB capabilities such as Change Data Capture, Materialized Views and Secondary Indexes. Most notably, it extends tablets to ScyllaDB’s DynamoDB-compatible API (Alternator). Although ScyllaDB was created as a Cassandra alternative, it has become increasingly popular as a DynamoDB alternative over the past two years.
What is tablet-based elasticity?
Did we just say tablet-based elasticity? Yes… and it doesn’t have much to do with iPads. The company’s tablet-based elasticity is a dynamic data distribution architecture that replaces the legacy static token ring approach (vNodes inherited from Apache Cassandra) that is designed to enable rapid, parallel and automatic scaling of a database cluster by breaking data into small, manageable units called “tablets”… so now you know.
ScyllaDB divides the database into small, fixed-size fragments called tablets, each roughly 5 GB in size. Each tablet is fully independent and can serve queries on its own. As soon as the first tablet moves to a new server, that server can begin offloading traffic from existing nodes. In practice, this happens about ten seconds after the server joins the cluster.
Close-to-the-metal, cheaper than AWS?
The team behind ScyllaDB say that it’s inherent efficiency comes from a close-to-the-metal design, features such as dictionary-based compression and file-based streaming and ongoing performance optimisations. The same team claim that ScyllaDB now guarantees that ScyllaDB X Cloud costs are 50% of DynamoDB costs, or lower.
TECHNICAL CLARIFICATION NOTE: As many readers will know, DynamoDB is a competing service to ScyllaDB. Amazon DynamoDB is a serverless, fully managed, distributed NoSQL database with single-digit millisecond performance at any scale. DynamoDB promises to overcome the scaling and operational complexities of relational databases and be able to handle huge spikes during Amazon Prime promotions that require multiple high-traffic Amazon properties and systems. This chunk of data engineering from AWS is said to continuously serve hundreds of customers with tables that have peak traffic of over half a million requests per second. It also serves hundreds of customers whose table sizes exceed 200 TB and processes over one billion requests per hour.
“ScyllaDB supported cross-cloud deployments, required a manageable number of servers, and offered competitive costs. Best of all, its API was DynamoDB-compatible, meaning we could migrate with minimal code changes. In fact, a single engineer implemented the necessary modifications in just a few days,” stated Todd Coleman, chief architect and technical co-founder at Yieldmo, a ScyllaDB user.
Prem Kumar Patturaj, senior manager at Freshworks, added, “DynamoDB’s item size limitation forced multiple steps: we had to split the data into multiple records, write it and then fetch it again. All of this logic ended up adding compute overhead. So, we migrated to ScyllaDB because these limitations don’t exist there.”
Architectural enhancements
ScyllaDB X Cloud is the next generation of ScyllaDB’s fully-managed database-as-a-service. It features architectural enhancements for greater flexibility and lower cost. Key X Cloud features include higher storage utilisation. Since servers can be brought online and start serving traffic fast, clusters can run at around 90% storage utilisation (compared to the more typical 70%). This reduces the number of servers required and lowers infrastructure costs.
“With storage autoscaling, [users] define the minimum CPU footprint for the cluster, and ScyllaDB automatically scales storage capacity to match. With mixed node sizes, the combination of shard-per-core execution and tablets allows clusters to mix very large nodes (for example, 128 cores) with small ones (as few as 2 cores). This enables high utilisation, simpler operations, and more granular scaling. With advanced compression: Up to 80% better compression reduces both storage usage and network transfer volume,” stated the company, in a technical product update.
What to think about ScyllaDB
To go up against AWS with some of its major technology claims is clearly no big thing for Scylla, the company has enough swagger and presence (at many technology shows throughout the year, in front of its customers, across special interest groups… and indeed at its own user symposium when it is staged) to carry it off. Deeply technical in nature, Scylla is one of those firms that plays in the vector technologies space for hard core software application developers who have specific competencies in data science.
All that… and Scylla is named after a man-eating female Greek monster from Homer’s Odyssey, what’s not to like?
