SQS

• Standard queues

  • at-least-once message delivery

  • duplicate messages can be delivered

  • best effort ordering

  • Better throughput then FIFO

• FIFO queues

  • exactly-once processing ,

  • No Duplicate messages

  • FIFO Order

  • Low throughput

• Deduplication of messages can be enabled by

  • Enable content-based deduplication

  • Explicitly provide the message deduplication ID

• SQS APIs

  • create_queue : Creates a new standard or FIFO queue

  • delete_message: Deletes the specified message from the specified queue

  • delete_message_batch: Deletes up to 10 messages from the specified queue

  • delete_queue: Deletes the queue

  • purge_queue: Deletes the messages

  • list_queues: list of your queues in the current region

  • receive_message: Retrieves one or more messages (up to 10)

  • send_message: Delivers a message

  • send_message_batch: Delivers up to 10 messages

  • change_message_visibility: Changes the visibility timeout message

  • change_message_visibility_batch: Changes the visibility timeout multiple messages

  • https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#id56

• Adjust ReceiveMessage parameter if consumer is not able to process incoming messages , ex: set ReceiveMessage to 10 to retrieve 10 messages at a time

• Short Polling :

  • Default SQS behavior , Get msg 🡺 send msg (returns immediately) ,

  • Responds even if no msgs in queue. Empty responses possibility is more

  • WaitTimeSeconds =0 or ReceiveMessageWaitTimeSeconds = 0

• Long Polling :

  • WaitTimeSeconds = 20 Wait till some msg arrive (or ReceiveMessageWaitTimeSeconds reaches) in queue then send msg

  • Responds only when at least one msg is in queue , sends an empty response only if the polling wait time expires (max 20sec)

  • Use case (recommendation)

    • Reduce Cost:

      • Reduce the num. of API calls you need to make

      • Reduce number of empty responses and false empty responses

    • If the new messages that are being added to the SQS queue arrive less frequently

    • Faster updates: Receive message as soon as they arrive

• VisibilityTimeout:

  • Period of time SQS prevents other consumers from receiving and processing the message which was already processed, (avoid duplicate processing)

  • Increase VisibilityTimeout (ChangeMessageVisibility) to reduce duplicate msgs being processed

  • Default is 30sec, min=0sec , max=12hrs

  • Used when your application takes a long time to process and delete (DeleteMessage) a message from the SQS queue

• Delay Queues:

  • Postpone the delivery of new messages to a queue for a number of seconds

  • Your consumer application needs additional time to process messages

• VisibilityTimeout Vs. Delay Queues

  • VisibilityTimeout : message is consumed once and then timeout is set for other consumers

  • Delay Queues : Message itself it delayed to be processed by any consumer for first time itself

• Dead Letter Queue:

  • Special queue for messages that can't be processed (consumed) successfully

  • Helps in debugging your application or messaging system

• MessageGroupIds

  • To process messages one by one, in a strict order (in particular message group)

• Auto Scaling SQS

  • Target tracking scaling policy (ApproximateNumberOfMessages)