Do Your Memories Empower or Restrict?