ETL caching sink Xslt loader All configuration loading Cache operation output (manually mark as cache-able, or auto-detect and configure based on instrumentation) - cache key pattern that includes converting dictionary to query string?
- do we need an app id that all instances for, say, SLS, use to prefix the cached items?
SNS integrated: let the application know that cache nodes have increased or decreased - need a generic app reconfigure construct
- or node auto-discovery in Java or PHP
Monitoring and alerts: integrated with CloudWatch - hits, misses,
- memory usage: not a useful metric since the cache will typically consume as much memory as it can
- evictions: more useful; indicates a need for more space / new node(s)
- AWS recommends specific metrics to monitor
Sharding: - consistent hashing: use a library that figures out hashing for you
Redis: - advanced data types
- atomic operations
- single threaded
- persistance
- read replicas
For logging sink, check out Kinetic.
Disaster Recovery - lower bandwith with replication vs backups
- backups to S3 with storage gateway or Cloudberry (or QBO)
- minimal db instance for pilot light;
|