The massively multiplayer online world is not geared toward the solo player. To a certain extent, it's logical: time is progress and multiple players can contribute more time; the tougher encounters have greater rewards, but take more power to conquer. And why have thousands of players congregated in a world if they aren't going to spend time together?
Thus, the creation of guilds, clans, corporations; the name does not matter as much as the concept. Players with similar goals, gameplay styles, and personalities band together to conquer content and explore virtual worlds. The general concept of "guilds" has become almost mundane in our community - they are must-haves in games, and naturally fall into various roles within a community. In my opinion, though, beneath the surface of guild creation and success is a complex structure based both on available gameplay and server population.
Guild composition and policy becomes increasing complex as you scale from casual family-style guilds to hardcore mandatory raid guilds. In general, all guilds must be large enough that members do not feel like they belong to an empty or undesirable group. Classes, or at least archetypes (tanks, healers, DPS, etc) must be balanced so that members can form groups for leveling - and for raids, classes must be fine-tuned to accomplish more difficult goals that require different strategies.
Guilds fill different niches. For a guild to be successful, its target audience - casual players, hardcore raiders, etc - must be large enough to furnish the guild. Numerous small family guilds can exist on the same server shard, though most often there are fewer high-end raid or PvP guilds. Raid guilds tend to fan out along the content line, and thus more raid content in a game means more room for raid guilds.
In my experience, there are often two guilds at the top of the mountain, one slightly ahead of the other: one is the number one Uber Guild, and the other is made of hardcore players who hate (or were rejected by) the number one Uber Guild. From there, guilds scale downward through content until they run out of elbow room - while a little competition is healthy, lack of targets will choke a guild into nonexistence. Here also is the virtue of instancing - if multiple guilds can progress through the same content at the same time, they can, for the most part, coexist. There is still competition for membership, but there is no risk of target choke.
As much as guilds are about competition, they are also about internal cooperation. Players work together toward a similar goal. As each player gains strength it increases the strength of the guild overall, however each player also counts on the guild for their own advances. One example of this in practice is the popular loot system, DKP (Dragon Kill Points) in which players are rewarding with 'points' for participating in raids. These points act as a currency and can be spent to buy items that drop off of raid targets; in effect, raiders take turns receiving gear and helping each other obtain gear that none of them could obtain on their own.
While guilds are fairly stable entities at the best of times, any game and server will have a certain amount of guild churn, where new guilds rise and giants fall. This too can be linked to game content: as new expansions are released, servers see an influx of population seeking new opportunities, and as time wears a game down, the population will slowly dwindle.
As said earlier, content choke is one cause of the death of guilds. Players often progress through content more quickly than developers produce it, and the natural fan of guild positioning will start to close with time. More casual raid guilds and those previously farther down the "ladder" will start to catch up with the most current high end content, and competition (assuming no instancing) will create a virtual sort of natural selection. The guilds that are able to move quickly on targets, and predict when they will be available, are the guilds that will survive.
Guilds also meet their end based on personality conflicts - also known as Drama. There is no way to please everyone, and eventually members will become disgruntled with loot methods, progression direction, other members, etc. This often leads to the rise of new guilds: disgruntled players will take their friends and leave an established guild to found one more in line with their ideals.
More casual guilds are in some ways more simple - there is no worry of target competition, for example - but these guilds deal with constant churn, as members decide to move into raid content. Casual guilds are often referred to as "grow-up guilds", where players will level - or grow - up in the guild, then move to a raid guild once they hit max level. Players seek companionship and company as they level, as well as friends around their level to group with in more difficult areas.
Our MMO lives are spent in daily monotony: we spend hours killing the same monsters in the same dungeons, weeks of adventuring profits are saved for a coveted new sword or breastplate. Guilds allow us companionship, teamwork, and a human element of excitement that would otherwise be missing in our games.