Near real time data processing using AWS Firehose, AWS Kinesis Analytics and AWS ElastiCache for Redis
One of the most used solutions to increase the performance of an application is by using a cache layer. The storage and database layer can be protected from throttling and intensive queries using a caching mechanism. In this article, we analyze if a standard cache solution is our best option all the time. Context We are working on a web application with a frontend and an API consumed by the front-end and other 3rd parties systems. We don’t have control on the 3rd parties, but every 2 months, the number of request increase with 200%. The current solution build by the team is written in .NET Core and runs inside AWS EC2 machines configured using AWS Beanstalk. In-memory caching is used for content that is often retrieved from the database (AWS RDS). New Gen The number of AWS EC2 instances increased, and to protect the database and backend, the team decides to disable stick session (session affinity) and migrate to a distributed cache solution. The team decides to use AWS ElastiCache