Kinesis

  • Handle Streaming data

Kinesis Streams

  • real-time data streaming service (DynamoDB does not provide real-time injection to DB)

  • massively scalable and durable can continuously capture gigabytes of data per second from hundreds of thousands of sources

  • website clickstreams, database event streams, financial transactions, social media feeds, IT logs, and location-tracking events, player-game interactions , stats updates

  • Single KCL worker can handle any number of shards

  • Number of KCL workers = < Num. of shards

  • When question is about maximum KCL workers then = num. of shards (1:1 ratio)

  • by default data records are only accessible for 24 hours, can be changed to max of 8760 hrs (365days)

  • Resharding: enable your stream to adapt to changes in the rate of data flow

    • Split shards: To increase the capacity of your stream, increases cost also.

    • Merge shards: To decrease the capacity of your stream, reduce cost

    • Hot shards: shards receiving more data

    • Cold shards: shards receiving less data