There are three main factors that I consider when evaluating a shoe's comfort: Cushioning (Softness), Stability (Firmness), and Midsole shape/support for the foot.
A really comfortable shoe, in my personal opinion, has a good ratio of cushioning AND stability while also having a supportive midsole shape.
Some shoes are really soft on feet, but lack the stability to back it up and can bottom out or feel too squishy as a result. Some shoes have a very firm and solid midsole, but also have little to no cushioning. Some shoes have an overall good ratio of cushioning and stability, but have a flat, unsupportive, midsole shape, which results in a shoe that lacks support for the entire foot.
For cushioning and stability, you are not necessarily looking for a shoe that has a 10/10 score in both factors. A shoe with 10/10 softness would have very little firmness and vice versa. You are looking for a good ratio between both scores, while also hoping that the shoe still scores relatively high in the factor you favor the most.
Another small note on the ratings: they are NOT like school grades, where a 5/10 = Failure. A 5/10 would mean that the shoe is average in that area. 3/10 and 4/10 would be below average and a 6/10 or 7/10 would be above average.
I rate all three factors out of 10 (ie: 7/10 for Cushioning, 8/10 for Stability, 7/10 for Midsole shape), as well as give a brief description of my opinion on comfort.
I will implement this new rating system in all my guides when I can!