ETL caching sink
All configuration loading
Cache operation output (manually mark as cache-able, or auto-detect and configure based on instrumentation)
- cache key pattern that includes converting dictionary to query string?
- do we need an app id that all instances for, say, SLS, use to prefix the cached items?
SNS integrated: let the application know that cache nodes have increased or decreased
- need a generic app reconfigure construct
- or node auto-discovery in Java or PHP
Monitoring and alerts: integrated with CloudWatch
- hits, misses,
- memory usage: not a useful metric since the cache will typically consume as much memory as it can
- evictions: more useful; indicates a need for more space / new node(s)
- AWS recommends specific metrics to monitor
- consistent hashing: use a library that figures out hashing for you
- advanced data types
- atomic operations
- single threaded
- read replicas
For logging sink, check out Kinetic.
- lower bandwith with replication vs backups
- backups to S3 with storage gateway or Cloudberry (or QBO)
- minimal db instance for pilot light;