All Administrators are granted moderation powers by default.
- Regularly review, take action on, and empty the flagged content at http://www.khanacademy.org/discussion/mod/flaggedfeedback
- Any posts with a flag count at our safety threshold (3 flags) will be immediately hidden until examined by a moderator.
- Our automated "low quality detection" system hides posts marking them with 3 flags. The system is not perfect, and we're needed to fish out valid posts.
- If a post has legit flags, delete the post. Guardian-deleted content is recoverable.
- If a post is ok, clear the flags/mark it "approved".
- Move questions that are in the comments section to the questions section and vice versa. If a post is both in the wrong category and very low quality, it can simply be deleted.
- Encourage users to move any problem reports, webapp and video suggestions, to the issue tracker until we have a way of tracking them separately in the discussion system. If the person has written a question, you can post an answer to direct them and then delete their question, which will hide it from the community.
- Edit content when necessary. If a post is useful but in all caps or has spelling mistakes or a dozen question marks, we correct it. If it is useful but asking for votes, we remove the part that asks for votes. You can also reword questions to make them better.
- Generally keep an eye out for wanted or unwanted behavioral trends so we can adjust the software to support our desired type of community.
- Send a message (via a user's profile) to users who have been participating in discussion but have made off-topic posts, such as chit-chat, repeated vote/badge seeking, or homework questions.
- "Hellban" users who are spamming the site or being abusive/offensive toward others, and who haven't already shown a significant capacity for making great posts. Khan Academy employee moderators have permission to hellban, you can contact them with recommendations of users to hellban. A hellban hides all of the user's current and future posts from the site, though when they are logged in, it appears that all of their posts are there.
As you moderate posts, you'll likely come up with your own workflows. Here are some ideas to get you started:
- Visit the "Flagged Feedback" queues at http://www.khanacademy.org/discussion/mod/flaggedfeedback. The "Low quality score" queue typically starts with posts captured by the automated system (they'll have three flags, and you can hover over that to see where the flags came from). The "Flagged" queue contains user-flagged items and is sorted by downvotes.
- Go to a subject's Community Questions page, like http://www.khanacademy.org/cs/d. There, you can march through the recent questions people have asked.
- Go to a particular video or program and read through the Questions, cleaning up off-topic stuff as you go.
When you find a questionable post, you'll likely want to open a new tab with that person's discussion profile, so you can see whether they've generated a bunch of off-topic posts. If so, you can delete their posts from their profile and, if appropriate, do a "Send Message" to them or go to the Guardians chat room and request they be hellbanned.
User profile controls
From a user's profile, you can not only browse their full discussion history, you can also send them a message (and if you're a KA employee, you can also hellban the user):
"Send message" allows you to send a message to the user. (The message will appear in the user's Discussion Notifications tab.) If a message has been previously sent to the user, the button will be orange:
The "Hellban" button allows you to stop a user from posting publicly on the site again. The user will not be aware of the hellban: anything she posts will only be seen by herself (and moderators). The "Hellban" button will also go back and delete all of the user's previous posts.
This feature is currently only available to Khan Academy developers.
The orange "Undo Hellban" button will appear if a user is currently hellbanned. Clicking it allows the user to resume posting publicly again, although their previous posts will stay deleted.
Example-based guide to moderating
Here, you can choose to moderate questions, answers, or comments, and you can choose from two feeds, "Low Quality Score" and "Flagged". Typically, the best bang-for-your-buck comes with moderating Questions -- deleting one bad question will also hide all of the answers associated with it. It is more work to determine whether individual answers or comments should be deleted.
Low Quality Score
"Low Quality Score" is a feed of all posts, ordered by the low quality score that our automated system assigns to each post. The higher the number, the more likely it is to be low quality. Everything in this queue is not bad, the automated system is very new and does not use advanced algorithms for determining quality. The automated system automatically applies 3 flags to posts above a certain low-quality threshold, so they are not posted to the site without explicit moderator action. You'll be seeing a number of these in this queue.
"Flagged" is a feed of posts that have been flagged by users or by the automated system. The feed is sorted with the lowest-voted posts first (this is a helpful but not perfect proxy for finding inappropriate content). You'll mostly see flags by users because the automated system, when flagging, never gives people a chance to vote, so they'll have 3 flags and 1 vote.
What to do in the queue
Moderators have several choices for what to do to questions in the queue. Here are some example posts and how we recommend you deal with them:
Occasionally, social comments may be part of an intentional graffiti spree, but usually it’s an innocent post by a new member. It’s likely not worth the time of opening a new tab to investigate the user’s profile.
- If the
post is over a month old, just delete and move on
- If it’s
more recent, paste the below message, or one of your own, into the answer
section and then delete the post (the user will still be able to see their own post and your response)
Welcome to Khan Academy, we’re glad
The questions area is designed for people that have questions about
the video they just watched. We want to make sure everyone that asks a question
can get it answered in a timely manner, so it’s important not to fill the
questions queue with social chat such as “Hi!” or “How are you?”
Your post has been removed to keep
the questions queue clean, but we encourage you to use it again when you do
have questions about any of the videos.
Thank you for your cooperation, and
we look forward to great contributions from you!
Some non-questions are worth moving
to the feedback section – others are not.
A good rule of thumb when is that a comment must meet
at least two
of the following three criteria in order to be worth keeping:
is coherent... I guess,
but isn't constructive or overtly positive.
||Statement of Confusion
The user has (presumably)
the video and needs more help. On one hand that’s exactly what the questions
section is designed for. On the other hand, the user has made little to no
effort to help you help them. The only way to assist the user
without more feedback is to essentially write an explanation of the entire
concept in a way that will be easier to understand than the explanation Sal
gave in the video – which is no easy task.
- If the post is dated, and doesn't have valuable community responses attached to it, delete and move on
- If it is a recent post, paste a message similar to the one below into the answer section and delete the question
I’m sorry you’re having trouble
understanding the concept. I would recommend re-watching the video and/or
watching the earlier videos for this concept. You may also want to try doing
some of the exercises. Remember, there’s nothing bad about getting some of the
answers wrong while you’re learning the material. Just make sure to review the
hints if you get stuck or if you get any of the questions wrong.
If you try all of these steps and you’re still having trouble, try to ask a
more specific question to get the best help.
There appear to be a number of feces enthusiasts within the Khan Academy community - and they like to make their presence known. Although these kinds of posts are relatively harmless, it's best to open up their profile and make sure there isn't anything more serious going on that hasn't been caught yet.
If this is as bad as it gets, send a message similar to the one below and delete the post
Remember to delete any other inappropriate posts on the user's discussion page before moving on
We're working hard to make sure that all of Khan Academy's users have a fun and safe environment to learn. In order to keep our discussions free of rude language, we recently had to delete one of your posts. In the future, please keep your questions clean and your feedback constructive. We encourage you to use the questions area again
when you have legitimate questions about any of the videos.
Thanks for your cooperation, and we
look forward to great contributions from you!
||All Caps question, Duplicate Answers
The question is all caps and could use cleanup, but is a legit question -- and it has good answers.
For the question itself, I'd edit it to fix the capitalization and make it clearer:
"Since like charges repel each other, why don't an atom's protons repel each other since they are very close together in the nucleus. What's holding them together?"
There are a couple of answers about the strong nuclear force from 8 and 7 months ago, and there's a duplicate answer from 2 months ago that doesn't add anything: "The strong nuclear force."
Delete the duplicate answer from 2 months ago.
|| Novel behavior in the Comments section of a CS program
Users are creating some interesting group behavior on CS programs. In this example, a group is dynamically forming and communicating with each other (albeit through a use we hadn't specifically intended).
As long as what's going on isn't really negative, I think it's ok to leave.
||When am I going to use this?
It might not seem like a particularly useful question, and in this case, it's possible that the asker is just trying to get a badge for using a timestamp.
However -- one of the motivation blockers people can have to learning is not understanding why something is useful. So the community's being able to explain and motivate the asker is a good thing.
Edit the post to fix the capitalization and remove the timestamp, it's irrelevant.
This person asked a lame question and then answered it him/herself, in some misguided attempt to get a badge or something.
||Comment in questions, requests votes
Basic attempt at vote-gathering. Also a comment in the question area.
||Question requests votes
Not a terrible question, but spelling errors and a request for votes. There are also good answers beneath.
Edit, fix up the spelling and nix the vote request:
"Do negative numbers, like -4 and -8, also have common multiples?"
||Question can use cleanup
Here's an ok question that can use just a little quick editing. It has a good answer, so I fix the question:
"How would I use multiplication instead of addition to find the perimeter?"
||Reasonable question, all caps
Edit the question to clean it up.
||Homework problem, followed by insults
Three months ago, this person asked a homework problem and it was tricky to understand what they meant. Then he wrote "Never mind". (Hover over the "3 months ago" to get an exact listing of what time the post was made, in the flagged queue these are sorted with lowest votes at the top, not in chronological order.)
Then jonathanrvr67 came along and was rude and insulting.
Open up jonathanrvr67's profile and see what kind of posts he has. He joined 6 months ago, hasn't used the site in months, has a few useless posts and these couple of offensive ones.
I'm torn between hellbanning jonathanrvr67 and just deleting this question, which will hide his offensive posts. I think I'll just delete the question and hope that jonathanrvr67 doesn't create more bad posts in the future.
||Comment in questions
This is a comment, not a question, and it's pretty useless.
||Meta-discussion about the site.
If the question is pretty new, you can refer the person to the discussion area at: http://www.reddit.com/r/Khan/ via an answer.
Then delete the question.
||Graffiti spawns more graffiti, and hellbanning
I try to visit the profile for ppyy2006 and no link appears. I click "Delete" but nothing happens. Ah, this user's account has been deleted, but this post remains. This is a bug on our end -- we need a way to delete these posts.
I visit the profile for mallymazz. Actually, I sent this user an email asking her to clean up her posts. But that was a good bit after this post. As long as she's not generating new off-topic posts, I'll leave her account alone. Actually, looking at it again, all of her posts are off-topic and she hasn't used the site in many months. Instead of individually deleting her posts, I go to her profile and choose "Hellban user".
If you're a non-employee moderator, you don't have the "Hellban user" button. For now, we've limited that to KA employees, but we may change that in the future. To recommend a person for hellbanning, email Nathan at firstname.lastname@example.org.
Straight-up homework problem. Also, no one has responded in 7 months with anything interesting. Delete.
||Crap about badges
I visit the profile of karmurphy and see that everything there is nonsense, pieces of web pages pasted in, more stuff about badges, just whatever. I don't see anything useful. 88 questions, 133 answers.
||Useless question, and a year old.
Looking at the profile, the user just drifted in and back out, this is their one remaining undeleted post.
Delete. Or, if you're feeling generous, Change to comment.
||Flagged question, but might be legit
Could be a legit question. I check out the video (clicking on the name of the video opens it in a modal player), and yes, it does actually make sense -- 'O' is a point of intersection inside the triangle.
I vote up the post, it's not a bad question.
Then I Clear 1 Flag. This takes the post out of the queue, and future flags by users will be ignored.
This one's from the low quality queue. It has 3 flags because it was auto-flagged by the system. That also means it's currently hidden. You can hover over "Clear 3 Flags" to find out where the flags came from.
I go to the person's profile. They just joined an hour ago, and their first post was this. I hit "Send Message" and welcomed them to the site but said that "bacon" wasn't helping anything.
||"I want the timestamp badge" post
Not helpful. I expect this person was just trying to get the "Use a timestamp" badge.
I go to his profile page and see that an hour ago, Drew (another moderator) already sent him a message. Cool!
If the person had framed it like, "do I add 80 or subtract 80," or something, maybe I might consider approving it. If the video it was on was about adding negative numbers.
However, the video it's on is an algebra video and pretty much unrelated to the question.
Looking at the profile, the person just joined, watched two minutes of video, and asked this question.
I don't send them an explicit message because I suspect they won't be asking another of these.
||Poorly worded question with good answer
This question isn't worded very well, but it's sort of on-topic, and the first answer is quite good.
I want to clean up the question, so I vote it up first. Then I choose "Edit" on the question and change it to "Is there a better method for doing harder addition problems?"
||Good answer to a non-question
However, I don't think it will have much future use for others on the site, it's mostly just a "you can do it!" message to the person who asked the question. The question asker has received a notification and can see the answer.
I'm going to Delete. I could possibly edit the question, but I still don't think the answer has a wide-enough use for the whole community.
||Interesting might-be-homework question
Here's a question from the Geometry "Community Questions" answer feed.
This seems like it might be a homework question, but it's not totally obvious. Also, the answer is interesting.
Ankit found this one. He answers the question and votes it up.
||Profile: useful answers + badge collecting
Most of this guy's posts are an attempt to collect badges. However, he also has some useful answers.
I do "Send Message" and mention his "off-topic posts and posts that just have the intent of collecting badges."
||Add me as your coach advertisements
Ankit sent him a message asking him to remove his posts, and then went and manually deleted them.
||Saying "Thanks" to an answer
We don't have a good way for people to send thanks to each other, so while their "thank-you" messages aren't explicitly answers, we let them stay there for now.
If this one had flags, you could approve it. Also since the person was thanking me. :)
||Junk posts for badge-gathering, plus other good posts
This user had set up a multi-account system for cheating to earn the "Detective" badge, 100+ questions with 3+ votes.
I messaged him a few days ago about removing his cheat posts, and he's removed almost all of them (I specifically mentioned Basic Addition, where most of them were).
I go ahead and manually delete the rest of his junk posts. However, he is a heavy user and gives good, thoughtful answers, so I don't want to hellban him.
I'm not sure what's going on here. This person has given good answers, but has also given us this nice piece of garbage. I almost wonder if someone has vandalized his account and made this post.
Actually, looking more, I see questions and comments, all from a single day, that are pure vandalism.
I delete the offensive post and send the user a message, telling him about the suspected vandalism and asking him to delete the post.
||Simple yes/no question
This yes/no question isn't particularly useful, and it's gathered over 30 typically junk answers.
||Paste junk into the text fields
This person set up an account with the nickname "ispamkhan". They only have a few posts so far, but they're mostly created by swiping out the text on the page and pasting it into the text field.
They don't have useful posts, and it seems unlikely they're going to start, given the nature of these.