RAM or CPU: Cost Optimization for Server Scaling

1. Why should we need Server Scaling?

At one point, our product is going to reach the computing power limit and can’t support new users. So, server scaling ensures that our system can handle increased traffic and workloads without compromising performance, maintaining a smooth user experience, avoiding downtime, and supporting business expansion.

✅In short, we’ll need server scaling when our business grows!

2. Vertical scaling vs Horizontal scaling

There are two main types of scaling: vertical and horizontal, and the right choice depends on your specific needs and goals.

🔰Vertically:

This means throwing more resources at the same server. In a cloud data center, this might be as easy as upgrading the server size for more CPU cores, memory, etc. This is the simplest method of scaling the hardware.

Some of the scenarios below that might need vertical expansion:

• If we need effective traffic management for a small business: vertical scaling may be the budget-friendly option for us.
• If we want to enhance one server’s performance without adding resources: use vertical scaling to improve an existing server’s performance without investing in new ones.
• If we don’t know how much traffic we’ll get
• If we need to take legacy applications further: boost the performance of legacy applications that don’t need high distribution or scalability. 
• If we want to avoid downtime as much as possible: as vertical scaling demands fewer upgrades, and less often, our application will be less likely to experience downtime.
• If we don’t have many capable engineers: vertical scaling may be the better option if we don’t have a strong team of engineers, as it focuses on enhancing a single server instead of several machines. 

🔰Horizontally:

Scaling horizontally means adding more servers to separate the load. This can reduce the load any single server receives. However, this will increase the complexity.

Some of the scenarios below that might need horizontal expansion:

• If we need higher performance: use horizontal scaling to spread the workload between several machines. That will boost the entire system’s performance. 
• If we need a flexible setup: as we have more than one machine with horizontal scaling, we can configure each one differently to boost overall efficiency.
• If we want to avoid a single failure point: if there are multiple machines, if one fails, the rest can compensate.
• If we have a microservices architecture: we can scale each microservice independently according to its demand. That can help a distributed system perform better overall.

3. RAM or CPU and Example Scenarios

Deciding whether to invest in more RAM or CPU depends on your workload:

• If your applications are running large databases or in-memory caches, increasing RAM will provide the best performance boost.
• If your applications are CPU-intensive, have complex logic, and use many algorithms, such as video encoding or scientific computations, upgrading the CPU will yield better results.

Understanding your specific workload requirements is essential for making the right choice.

Which is more important? Food or Water?

Well, you really can’t live if we don’t have either of those. We need both of those to survive. Same with servers, having a CPU without RAM is pointless as well as having RAM with no CPU.

⭐Upgrading RAM gives visible improvement, but in my experience, after 16GB, you stop seeing improvement. Also, the MHz of the RAM DOES affect performance.

⭐CPU will visibly affect performance. Though it varies slightly based on the quality of our GPU, a CPU with higher clock speeds and core/thread count will outperform CPUs with lower clock speeds and core/thread count.

Focus on RAM:

If we’re serving simple, static websites we want more RAM. The idea would be that we want as many pages as possible loaded into the OS cache so that they’re served directly out of RAM. Static content is the simplest use case for any web server.

Focus on core-to-RAM ratios:

when we talk about building application servers, say that a given application needs 2G/Core or 12G/core to perform optimally. But we can’t know that unless we run it for a while -> the best option only be to benchmark it, properly benchmarking an application more frequently to find the right ratios and amount of your business’s needs.

Focus on CPU:

Finally, we could have a trivial app that you need to scale a lot. For example, consider we wrote an app that executes `uptime` on the server whenever we hit www.our-website.com/uptime. In that case, we’d want enough cores that can handle the expected load of /uptime requests, but we need very little RAM. Just enough to boot the OS and run a few hundred copies of a trivial binary.

Anyway, both affect performance and are necessary, it all depends on how the application you’re running is designed.

4. Let’s Picking the Best Server for our Business

In conclusion, choosing the right server wasn’t easy, we should evaluate current and future needs, and consider many things like expected traffic growth, application demands, and budget. Assessing these factors will help us pick the best server setup for our business.

By understanding the importance of server scaling and knowing when to optimize for RAM or CPU, we can make the right decisions, and ensure the system runs efficiently as our business grows.