Guilds are flexible in GW2. I say that because you can actually join 5 Guilds per account. People will often join a guild for every type of content they play. Perhaps one for PvE, one for WvW, one for role-playing, etc. I will not be covering how to start a guild/upgrade it. The wiki is already a one-stop shop for that information.
When selecting a guild, there are certain benefits to joining one that has been well established. Upon opening the guild menu (“G” by default), you’ll be able to see the list of guilds that you’re in on the left. You must check a box next to a guild’s name to “Represent” them. Each character on your account has its own representation saved, so remember to rep on each character. That means that your guild tag will show up to other players. You’re able to represent one guild at a time and there is no penalty for switching around, although some guilds require you to rep them a certain percentage of the time. This is a bit archaic, as guilds are no longer rewarded the same way as they were in the past for having more represented members and I would advise that you stay away from a guild that treats you poorly for not representing them.
Guild Hall - If properly upgraded, it can provide resource nodes for you to collect on a daily basis, a combat arena, a quick access point to Lion’s Arch, a variety of free boosts, a place to practice your creativity by building custom structures/jumping puzzles, amongst other things. Also, if your guild has the upgrade, there’s a merchant that will sell you gathering supplies/salvage kits. The Guild bank allows you to access your personal bank! Lastly, if you are in two or three guilds that have different guild halls (there are currently three: Lost Precipice,
Guild Mission - If your guild hosts these, you may participate to help the guild earn favor and you will earn guild commendations, which may be exchanged for a variety of gear and other items. I recommend checking the wiki link I provided to see what the vendors offer in exchange for commendations. I will say that it’s an easy way to get ascended trinkets, especially if you don’t own the expansions or Living World Seasons 3 or 4. Also useful to get ascended recipes.
Guild Portal - Quick and easy way to access a few areas, including Lion’s Arch. I often travel to my guild hall and use it to save a bit of money.
Guild Enhancements - Boosts that last for 24 hours. The amount depends on the number of enhancements the guild has unlocked, up to a max of 10% for most categories. Exceptions include crafting critical chance (bonus crafting xp) and critical harvesting (chance to get rare materials), which offer a 20% boost.
Guild Arena - An area for you and your guildies to duel. Uses your PvE gear, skill balance, and level (WvW/sPvP skills are sometimes balanced differently). If you host a party and invite someone, technically you can bring them into your guild hall and fight them there.