Original Anime Document: https://docs.google.com/document/d/1Ga2kJB-bzRPQMhiHDSlHSum3YGVHRIb6rwM2qhC7WPM/edit?usp=sharing
This document is no longer being updated, but still holds some extra information.
https://sites.google.com/view/07012022/anime-list
To clarify the rating categories:
Story: The show's concepts, setting, timeline of events, and execution of the show's concept. If fanservice brings down the quality of the show, you will see it in this category's rating. This also includes the rating of the ending if it has one.
Characters: Obviously the characters, but also their relationships, interactions, and developments. Character design affects this category but will mostly affect the visuals category.
Pacing: How consistent the show is in quality. Pacing is very important in shows, and some shows have varying quality between episodes and seasons. Pacing to me is basically how bored I am while watching it.
Audio: This includes the sound design, voice acting, openings, endings, and background music of a show. It rates how well the music and sound fits the scene, and how well it creates an atmosphere. Music to me can be an experience and if it can create a memorable experience in a scene, then those tracks will stick out to me a lot and boost the score by a lot. The voice acting can add a lot to the characters, but I'm still gonna put it in this category.
Visuals: Includes character design, scenery, pretty much anything you are seeing on screen. There are few shows that have "bad" visuals, but the ones that try to do something different that looks good will get higher ratings. Badly executed CG is a good example of how the visuals can go down in score.