Trending Games | Final Fantasy XIV: A Realm Reborn | Guild Wars 2 | EverQuest | Landmark

  Network:  FPSguru RTSguru
Login:  Password:   Remember?  
Show Quick Gamelist Jump to Random Game
Members:2,920,565 Users Online:0
Games:760  Posts:6,311,893
Sony Online Entertainment | Official Site
MMORPG | Genre:Fantasy | Status:Development  (est.rel 2014)  | Pub:Sony Online Entertainment
PVP:Yes | Distribution:Retail | Retail Price:n/a | Pay Type:Free | Monthly Fee:n/a
System Req: PC | Out of date info? Let us know!

EverQuest Next Forum » General Discussion » How will the game handle the huge amount of data for changing world?

2 Pages « 1 2 Search
39 posts found
  Mardy

Novice Member

Joined: 9/01/06
Posts: 2222

9/05/13 7:16:27 PM#21

How will the game handle it?  Through lag of course :)

 

Planetside 2 is free to play, go download it, and head to the nearest hot spot.  You'll see how your computer will handle Forge Light engine.  

EQ1-AC1-DAOC-FFXI-L2-EQ2-WoW-DDO-GW-LoTR-VG-WAR-GW2-ESO

  Ulorik

Apprentice Member

Joined: 9/12/11
Posts: 182

 
OP  9/17/13 5:58:28 AM#22
Originally posted by SnarlingWolf
Originally posted by Ulorik

Tech noob here. Someone might have a very easy to understand answer, but I'm scratching my head at this:

A changing buildable and desctructible  world sound really exciting, however I was wondering. How will the huge amount of data be handled that you need to get to each player if you want to experience in realtime all these changes around you,  crumbling walls, detroyed bridges, dungeons being dug, houses and villages being erected and destroyed. That on top of your mob movement and fighting, possibly PvP etc, etc.

 

This sound like a huge step up in data quantity to be moved around compared to "traditional" MMO's

I personally think people are over thinking marketing hype speech.

 

I would bet most of it will be state changing, not actually world changing. This means that there will be set states an area can be changed to through actions, not a freedom to change it how you want. So if enough people do this one thing, this part of the area changes to State 2 from State 1. Not you run in and dig a giant hole in the ground somewhere and watch things fall into it.

 

So a bridge will have two states. Built and destroyed. The game just swaps between the two. The physics are easy enough as the built model has different physics from the destroyed. That part really is not complicated at all.

Yes, thanks for all the answers. I think the above answer is hitting the spot. I had understood the Voxel engine really in that way that people would be in-game manipulating Voxels that make up the Landscape...hence my bewilderment about data loads. If you think about different states just to be sent to the client it of course is manageable.

  drbaltazar

Advanced Member

Joined: 3/28/07
Posts: 7987

9/17/13 6:04:49 AM#23
There is a game using this technology (rush if I recall)limitless world almost.)but it aint an mom.the main issue anmmo face is concurrent amount of player on a small square.like me in ff14 I walk around nothing in sight?my ms go from 14 to 20 .I meet people it drop to 40 to 60 ms!This is why I hope ms enabled a version of donnybrook for Xbox one!
  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/17/13 6:32:37 AM#24
Originally posted by SnarlingWolf
Originally posted by Ulorik

Tech noob here. Someone might have a very easy to understand answer, but I'm scratching my head at this:

A changing buildable and desctructible  world sound really exciting, however I was wondering. How will the huge amount of data be handled that you need to get to each player if you want to experience in realtime all these changes around you,  crumbling walls, detroyed bridges, dungeons being dug, houses and villages being erected and destroyed. That on top of your mob movement and fighting, possibly PvP etc, etc.

 

This sound like a huge step up in data quantity to be moved around compared to "traditional" MMO's

I personally think people are over thinking marketing hype speech.

 

I would bet most of it will be state changing, not actually world changing. This means that there will be set states an area can be changed to through actions, not a freedom to change it how you want. So if enough people do this one thing, this part of the area changes to State 2 from State 1. Not you run in and dig a giant hole in the ground somewhere and watch things fall into it.

 

So a bridge will have two states. Built and destroyed. The game just swaps between the two. The physics are easy enough as the built model has different physics from the destroyed. That part really is not complicated at all.

They are using a voxel engine, which allows much more detailed destruction and terraforming. Minecraft is a good example of a voxel based game (mostly because everyone knows it).

Your example is related to traditional 3D engines and not applicable here.

The biggest problem with voxels is that making moving voxel structures is practically impossible unless the SoE codewizards have figured out some really awesome stuff.

 

  Kyllien

Novice Member

Joined: 3/22/13
Posts: 315

9/17/13 7:03:56 AM#25
Originally posted by tom_gore
Originally posted by SnarlingWolf
Originally posted by Ulorik

Tech noob here. Someone might have a very easy to understand answer, but I'm scratching my head at this:

A changing buildable and desctructible  world sound really exciting, however I was wondering. How will the huge amount of data be handled that you need to get to each player if you want to experience in realtime all these changes around you,  crumbling walls, detroyed bridges, dungeons being dug, houses and villages being erected and destroyed. That on top of your mob movement and fighting, possibly PvP etc, etc.

 

This sound like a huge step up in data quantity to be moved around compared to "traditional" MMO's

I personally think people are over thinking marketing hype speech.

 

I would bet most of it will be state changing, not actually world changing. This means that there will be set states an area can be changed to through actions, not a freedom to change it how you want. So if enough people do this one thing, this part of the area changes to State 2 from State 1. Not you run in and dig a giant hole in the ground somewhere and watch things fall into it.

 

So a bridge will have two states. Built and destroyed. The game just swaps between the two. The physics are easy enough as the built model has different physics from the destroyed. That part really is not complicated at all.

They are using a voxel engine, which allows much more detailed destruction and terraforming. Minecraft is a good example of a voxel based game (mostly because everyone knows it).

Your example is related to traditional 3D engines and not applicable here.

The biggest problem with voxels is that making moving voxel structures is practically impossible unless the SoE codewizards have figured out some really awesome stuff.

 

The magic will be in procedural generation on the client, compression, and pre-stored image data on the client.

Only the voxels on the surface need to be rendered on the client computer. The interior of the object can be procedurally generated on the client. 

There will likely be an insane amount of repetition, the more repition the smaller the compressed data is. 

Since your client will be able to build the exact same thing as any other player; likely your client already has all the data it needs to be able to render the image on the client.  The trick is in figuring out how to put it together.

  Electro057

Advanced Member

Joined: 3/05/09
Posts: 545

9/17/13 7:13:55 AM#26

With magical theoretical fairies, goofer dust, cyber bunnies, and spriggans! 

Though I do firmly believe, like everything in reality compared to fairytales...It will disappoint without fail. I mean Second Life allows users to create and update and upload content, meshes, textures and build land....And it's still horribly laggy, demanding of computer and network resources....

Though this game won't let us import anything new and fresh, it'll already all be on the client, no new textures or meshes that we create from a 3D design program....So in that aspect it just needs to tell the client to arrange resources it already has readily available, which is much less complicated. 

--Custom Rig: Maker's Forge---
3.0 Ghz AMD Phenom II X6 1055T Processor (Six Cores, 3.0 Ghz Each)
8 Gigs DDR3
Radeon HD 6990 4 gigs of dedicated ram.
Windows 7 Ultimate

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/17/13 8:06:37 AM#27
Originally posted by Kyllien
Originally posted by tom_gore
Originally posted by SnarlingWolf
Originally posted by Ulorik

Tech noob here. Someone might have a very easy to understand answer, but I'm scratching my head at this:

A changing buildable and desctructible  world sound really exciting, however I was wondering. How will the huge amount of data be handled that you need to get to each player if you want to experience in realtime all these changes around you,  crumbling walls, detroyed bridges, dungeons being dug, houses and villages being erected and destroyed. That on top of your mob movement and fighting, possibly PvP etc, etc.

 

This sound like a huge step up in data quantity to be moved around compared to "traditional" MMO's

I personally think people are over thinking marketing hype speech.

 

I would bet most of it will be state changing, not actually world changing. This means that there will be set states an area can be changed to through actions, not a freedom to change it how you want. So if enough people do this one thing, this part of the area changes to State 2 from State 1. Not you run in and dig a giant hole in the ground somewhere and watch things fall into it.

 

So a bridge will have two states. Built and destroyed. The game just swaps between the two. The physics are easy enough as the built model has different physics from the destroyed. That part really is not complicated at all.

They are using a voxel engine, which allows much more detailed destruction and terraforming. Minecraft is a good example of a voxel based game (mostly because everyone knows it).

Your example is related to traditional 3D engines and not applicable here.

The biggest problem with voxels is that making moving voxel structures is practically impossible unless the SoE codewizards have figured out some really awesome stuff.

 

The magic will be in procedural generation on the client, compression, and pre-stored image data on the client.

Only the voxels on the surface need to be rendered on the client computer. The interior of the object can be procedurally generated on the client. 

There will likely be an insane amount of repetition, the more repition the smaller the compressed data is. 

Since your client will be able to build the exact same thing as any other player; likely your client already has all the data it needs to be able to render the image on the client.  The trick is in figuring out how to put it together.

The compression won't work very well outside the release when people start changing the landscape. Myself I'm guessing they are using some clever recursion on the voxels - in effect having another voxel space inside each "master voxel", etc. where it's needed. In fact, that is probably the only way they can have different size voxels without the memory requirements for mapping the whole world from exploding. Basically, most structures would be comprised of X number of big blocks and then a smaller number of those would contain a sub-voxel space, of which a number would contain a third level of sub-voxels, etc.

In effect, the more detailed a structure would be, the more memory it would require to load. Now, I'm also guessing their smoothing algorithm does a ton of this procedurally within the client so that the server doesn't need to hold all that data.

What you would see when entering inside a detailed structure is first a very rough scetch that will then (de)form into the final structure real-time. We will probably be watching a loading screen at that point, though. The trick to stop this happening while in-game is to load the space near the player as far as he can see and then some, and unload it as he moves away from those areas. In effect we would only get loading screens when teleporting from one spot to another, and even then if the system is smart enough it could pre-load the teleport destination as we get close to the portal (or w/e).

What I'm wondering is that how they plan to implement moving structure parts, such as windmill wings, into the voxel-based structures so that they will look to be part of it and will behave realistically once the voxel structure gets destroyed.

 

  lizardbones

Hard Core Member

Joined: 6/11/08
Posts: 10953

I think with my heart and move with my head.-Kongos

9/17/13 9:24:49 AM#28

According to the VoxelFarm website, they are converting the information between voxels and polygons on the fly. They aren't storing or transmitting the same amount of raw information as a game like Minecraft.

I can not remember winning or losing a single debate on the internet.

  Anakami

Novice Member

Joined: 2/28/12
Posts: 101

9/17/13 11:23:50 AM#29
Wasnt that Golem that destroyed the building a huge walking voxel structure? Just wondering, because I assumed that the golem too was made of voxels.
  Grahor

Apprentice Member

Joined: 5/08/11
Posts: 854

9/18/13 5:29:06 AM#30

People, you still won't get realistic behavior. No physics, no debris except for particle effects, no wall falling because someone removed the bottom of it.

 

Yes, it's theoretically possible to do those things with voxel engine; it's practicly impossible to do it in real time in multi-user environment. 

 

And no, no "code wizards" may "find a way", laws of informatics are not flexible: you can trade something for something, but you can't have everything. You can have modified terrain with huge blocks and small number of users (see Minecraft), but not all together.

 

My guess, it'll be mostly indestructible environment, quickly healing, no physics, lots of "state changes" the usuall MMORPG way.

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/18/13 7:05:26 AM#31
Originally posted by Grahor

People, you still won't get realistic behavior. No physics, no debris except for particle effects, no wall falling because someone removed the bottom of it.

 

Yes, it's theoretically possible to do those things with voxel engine; it's practicly impossible to do it in real time in multi-user environment. 

 

And no, no "code wizards" may "find a way", laws of informatics are not flexible: you can trade something for something, but you can't have everything. You can have modified terrain with huge blocks and small number of users (see Minecraft), but not all together.

 

My guess, it'll be mostly indestructible environment, quickly healing, no physics, lots of "state changes" the usuall MMORPG way.

Of course you can have those. Using Minecraft as metrics is bad because even Mojang has agreed the multiplayer code of Minecraft is rubbish.

Sure the system requirements for this game might be high-ish, but then again the cartoony art style will probably remedy that somewhat.

Your "guess" would be a pretty traditional MMO with nothing groundbreaking to offer. I believe SOE can do better. They have genuinely surprised me with PlanetSide and PlanetSide 2. When the original PlanetSide was announced, everyone was going "no, that's impossibru!", but they made it work, with 2003 internet connections nonetheless.

If they promote the game with "fully destructible terrain", would it not be a bit misleading if most of the terrain wasn't destructible after all?

 

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/18/13 7:09:38 AM#32
Originally posted by lizardbones

According to the VoxelFarm website, they are converting the information between voxels and polygons on the fly. They aren't storing or transmitting the same amount of raw information as a game like Minecraft.

Now this is exactly what I mean when I talk about the code wizards and stuff (be they working at SOE or elsewhere).

We DO have the computing power to do amazing stuff. It just needs to be used in a clever way.

  Grahor

Apprentice Member

Joined: 5/08/11
Posts: 854

9/19/13 8:27:24 AM#33
Originally posted by tom_gore

If they promote the game with "fully destructible terrain", would it not be a bit misleading if most of the terrain wasn't destructible after all?

 

I'll just answer this: no. Terrain would be fully destructible, just not all terraing and not everywhere, not all the time and not permanently. Otherwise, it would be fully destructible terrain. It would not be aligned with all the expectations, but it would not be a lie.

 

As for VoxelFarm - it's an engine, or at least one of engines, which is used in EQN. By itself, it's a MARVELOUS engine that can do wonders with voxels and is the product of work of one man, but it doesn't do miracles, and it wasn't created with mmorpgs in mind - it's a voxel building instrument, "a farm" where you grow voxels, not an engine for a game in itself. 

 

It does not "turn voxels into textures", it shows voxels using textures, but the voxel data have to be there and to be transmitted between server and computer in voxel form (with the exception of designer-designed objects that can be transmitted as metadata, but once you change them in any way, they are not the same objects anymore)

 

Aaaargh. Well, I surrender. :) I hoped to talk a bit of specifics, but apparently I'm the only one who is even trying. :)  We'll just have to see. And the one who's expectations are farther from truth will have to post an admission of wrongness in a form of "well paint me pink and spank me bottom!" :)

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/19/13 8:49:50 AM#34
Originally posted by Grahor
Originally posted by tom_gore

If they promote the game with "fully destructible terrain", would it not be a bit misleading if most of the terrain wasn't destructible after all?

 

I'll just answer this: no. Terrain would be fully destructible, just not all terraing and not everywhere, not all the time and not permanently. Otherwise, it would be fully destructible terrain. It would not be aligned with all the expectations, but it would not be a lie.

 

As for VoxelFarm - it's an engine, or at least one of engines, which is used in EQN. By itself, it's a MARVELOUS engine that can do wonders with voxels and is the product of work of one man, but it doesn't do miracles, and it wasn't created with mmorpgs in mind - it's a voxel building instrument, "a farm" where you grow voxels, not an engine for a game in itself. 

 

It does not "turn voxels into textures", it shows voxels using textures, but the voxel data have to be there and to be transmitted between server and computer in voxel form (with the exception of designer-designed objects that can be transmitted as metadata, but once you change them in any way, they are not the same objects anymore)

 

Aaaargh. Well, I surrender. :) I hoped to talk a bit of specifics, but apparently I'm the only one who is even trying. :)  We'll just have to see. And the one who's expectations are farther from truth will have to post an admission of wrongness in a form of "well paint me pink and spank me bottom!" :)

Fair enough. Yes I don't expect everything everywhere to be destructable and irreversibly so. That would be just silly. It doesn't even work in Minecraft and thus they have protection tools to protect people's building areas. I still expect to be able to go out of the hotspots, pick up a shovel and dig my way down to whatever they call their "underdark". Could be I'm wrong, but at least this is what they have been advertising.

And yes I'm quite aware how much data voxel engines have to handle, but as said several times before in this chain, they will probably resort to procedural generation/deformation of the terrain. In effect, when you hit ground with your AWESOME DIGGING SPELL (we'll just call it ADS from now on), the deformation of the terrain will be exactly the same every time, if you hit the exact same spot.

It means that when something is deformed, the client and server do not need to send each other information of which voxels were changed. Instead they only exchange the information about the event. What effect hit and where, and the calculations are made locally and will end up in an identical result.

Now of course there will still be a lot of data that needs to be transferred between the server and the client (mostly from the server to client) when you move up to an area the first time after it has been changed since your last visit. Whether it's faster to send just the raw voxed data or the just the effects (as described above) depends on the amount of effects and the amount of data that has been changed. And again, as long as the terrain is smartly composed of big chunks which divide into smaller chunks as needed, the data for a huge quarry mined into a previous flat spot would actually be quite small.

 

  Grahor

Apprentice Member

Joined: 5/08/11
Posts: 854

9/19/13 9:34:17 AM#35

Yeah, ADS being an object in itself was how I've thought it out myself; and I have no doubt that you'll be able to dig a hole with ADS into the mine below (which, I believe, will be an instance generated as an event on hole deep enough, instance accessible to others through the same hole but NOT through the hole dug nearby - even though people would dogpile me for this) - but you won't be able to dig the same hole through the wall of the keep; you would not be able to dig a lot of holes around a chunk of earth or a tree and see it fall down; you would not be able to dig through a column and see it falling down - in short, such ADS-as-an-object that generates change of landscape on client machine is incompatible with physics.

 

But now imagine 50 people together casting their ADSses on the terrain. How long before the data layer of the terrain will be overloaded? Say, 50 people 1 cast every 10 seconds, 300 ADS events a minute, 6000 ADS event in 20 minutes. Any new player entering area will have to get 6000 ADS events which will have to be processed by his machine immediately. Is it possible? I don't know, but I have doubts.

 

And what if it's 100 people and 40 minutes?

 

Also, monster pathfinding and ADS - you dig a series of holes around the mob, pathfinding algorithm goes sulk in the corner. Easy to abuse, no?

 

I think there would be throngs of limitations on where and when you can use ADS.

  Grahor

Apprentice Member

Joined: 5/08/11
Posts: 854

9/19/13 9:45:57 AM#36

P.S. VoxelFarm creator writes about networking VoxelFarm:

 

"Network tests are good at showing why some stuff does not work. But when the results are good it does not really mean anything. The real network is so complex you cannot replace it by any model. In this case results are as good as any network test can be at this stage. There is very little overhead from the thread and connection management, which is what I was looking for.

 

While this is good news and by all means necessary, the real bottleneck comes from how any application using this engine chooses to store and process information. So again what you are seeing here is just a brick. You could create many different houses with it.

 

You could do it like Minecraft servers do, have everything including procedural generation run in the server. You could do like this particular demo does, where user-created content is stored in a server and everything else remains client-side. And you could have solutions in-between, for instance have some custom server-side generation which is merged later with the rest of the client-side generation."

 

So far nobody knows how it will work. Also, keep in mind that VoxelFarm's voxels are 30x30x30 cm - 1/3 of a meter. They could be made smaller, but making them smaller means more data - cutting them by half generates 8 times more data.

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/20/13 1:41:36 AM#37
Originally posted by Grahor

Yeah, ADS being an object in itself was how I've thought it out myself; and I have no doubt that you'll be able to dig a hole with ADS into the mine below (which, I believe, will be an instance generated as an event on hole deep enough, instance accessible to others through the same hole but NOT through the hole dug nearby - even though people would dogpile me for this) - but you won't be able to dig the same hole through the wall of the keep; you would not be able to dig a lot of holes around a chunk of earth or a tree and see it fall down; you would not be able to dig through a column and see it falling down - in short, such ADS-as-an-object that generates change of landscape on client machine is incompatible with physics.

 

But now imagine 50 people together casting their ADSses on the terrain. How long before the data layer of the terrain will be overloaded? Say, 50 people 1 cast every 10 seconds, 300 ADS events a minute, 6000 ADS event in 20 minutes. Any new player entering area will have to get 6000 ADS events which will have to be processed by his machine immediately. Is it possible? I don't know, but I have doubts.

 

And what if it's 100 people and 40 minutes?

 

Also, monster pathfinding and ADS - you dig a series of holes around the mob, pathfinding algorithm goes sulk in the corner. Easy to abuse, no?

 

I think there would be throngs of limitations on where and when you can use ADS.

Instancing the dungeons would be a workaround. Could be they will need to resort to such, let's hope they will not.

About those massive amounts of ADS events, it would obviously be easier at that point to just send the data of the final state after those events. As said, calculating the amount of data needed to send either way is quick and the server can decide which method to use.

Pathfinding AI has and can be fooled even without voxel engines and ADS. I assume they will also put in countermeasures for this, such as the monsters becoming invulnerable if they cannot find a path to the player. Pretty standard in MMOs, even if it's just a workaround.

 

  tom_gore

Apprentice Member

Joined: 2/27/09
Posts: 1806

9/20/13 1:45:40 AM#38
Originally posted by Grahor

So far nobody knows how it will work. Also, keep in mind that VoxelFarm's voxels are 30x30x30 cm - 1/3 of a meter. They could be made smaller, but making them smaller means more data - cutting them by half generates 8 times more data.

SOE has some pretty good networking guys. They made Planetside 2 work, which is quite an amazing feat compared to how other companies struggle with 32v32 players. I have faith :)

About the voxel size, as I suggested smaller voxel size can be done locally by recursively creating sub-voxelspaces inside the bigger voxels. Using this method the "master" voxel size can be very large and enable transmitting data of uniform masses of material (such as a ginormous pile of dirt or all that air in the world) very compact. There is a mod that does this for Minecraft, enabling you to have more detail on places you need it, without totally exploding the world data size.

You're absolutely correct about the physics of voxel objects though. I have no idea how they will solve that, as I wondered earlier how to make windmill wings, for example. I guess they could combine traditional polygon models, as they are already doing with player characters. Wasn't there a mention that the voxelfarm engine can be used to fluidly transform between polygon models and voxel models?

It's interesting to see how they handle it and how much they will eventually need to "water it down".

 

  dandurin

Advanced Member

Joined: 5/19/04
Posts: 478

9/20/13 1:57:33 PM#39


Originally posted by tom_gore

Originally posted by Grahor So far nobody knows how it will work. Also, keep in mind that VoxelFarm's voxels are 30x30x30 cm - 1/3 of a meter. They could be made smaller, but making them smaller means more data - cutting them by half generates 8 times more data.
SOE has some pretty good networking guys. They made Planetside 2 work, which is quite an amazing feat compared to how other companies struggle with 32v32 players. I have faith :)

They also made the EQ2 incremental downloader which lets you get into the game on an uninstalled machine in 3 minutes. So I don't buy at all the argument some are making that you won't be able to run around Landmark and see other people's creations. You'll just get continual background updates streamed over time. Yes, the development plots will likely be seperated a bit, but so what.

About the voxel size, as I suggested smaller voxel size can be done locally by recursively creating sub-voxelspaces inside the bigger voxels. Using this method the "master" voxel size can be very large and enable transmitting data of uniform masses of material (such as a ginormous pile of dirt or all that air in the world) very compact. There is a mod that does this for Minecraft, enabling you to have more detail on places you need it, without totally exploding the world data size.


I don't know where Grahor is getting his information, Voxel Farm typically uses 0.1 meter resolution on it's voxels, which is plenty for Landmark.

You're absolutely correct about the physics of voxel objects though. I have no idea how they will solve that, as I wondered earlier how to make windmill wings, for example. I guess they could combine traditional polygon models, as they are already doing with player characters. Wasn't there a mention that the voxelfarm engine can be used to fluidly transform between polygon models and voxel models? It's interesting to see how they handle it and how much they will eventually need to "water it down".  
 
Voxel Farm just contributes polygons built from voxels to Forgelight. They can add whatever they want on top of that. Yes, everything dynamic like windmills will use traditional polygonal modelling, as will really fine-grain models like chandeliers, flowers, etc.

2 Pages « 1 2 Search