Read-after-write consistency [1]
Some times API need to read from slave to distribute the load, but during replication lag the data on the slave can be eventually consistent so we might need to think of a way to decide whether to read data from master or slave. This becomes more complex when we work on API powering multiple types of devices.
Monotonic Reads [1]
Some times we might need to query same slave instead of random slave in case same user is asking for same data. So that any consecutive query wont go to slave with significant lag and user sees consistent data always. This will be better than eventual consistency like pseudo-strong consistency.
How can service decide which slave to query data since this routing is not handled at application layer rather at load balancer layer?
Usually not done at application level, though guarantee can be provided by "transaction" implementations.
Between Services [1]
REST
Pros
Easy Debugging
Many tools
RPC
Preferred for communication between services in same organization boundary.
Tools
Documentation
Points to consider [1]
Size of the serialized data
Serialization & deserialization compatibility between server and client in different platforms and languages.
Documentation of the data
Forward & Backward Compatibility
Versioning of schema
Textual Format
JSON
XML
CSV
Binary Format
Apache Thrift
Protocol Buffers
Apache Avro
Parquet
With server-side applications you may want to perform a rolling upgrade (also known as a staged rollout), deploying the new version to a few nodes at a time, checking whether the new version is running smoothly, and gradually working your way through all the nodes. This allows new versions to be deployed without service downtime, and thus encourages more frequent releases and better evolvability.[1]
Designing Data Intensive Application by Martin Kleppmann. link