Server population over time

betame · 2018-12-30 02:31:58

If you already grabbed that data, cool.
CSV works as a standard
either PM me or a link (google drive or?) for everyone here

thundersen · 2018-12-30 12:33:22

betame wrote:

If you already grabbed that data, cool.
CSV works as a standard
either PM me or a link (google drive or?) for everyone here

I don't have it as CSV yet. I will try to create it in the next few days and will publish it somewhere when ready.

thundersen · 2018-12-30 15:28:48

betame wrote:

I wish I had data on the average ratio of birthers.

Here we go. I've calculated it for servers 1,2,3 from December 14 through 28.

The mean fertile mom ratio over all servers is 0.275, which is pretty close to your estimate of 25%!

         fertile_moms  ratio_fertile_moms  player_count
count  21601.000000        15521.000000  15521.000000
mean      36.186195            0.274950    141.778860
std        9.284623            0.039465     29.156598
min        0.000000            0.005760     59.600000
25%       29.000000            0.250730    121.625000
50%       36.000000            0.273504    145.500000
75%       43.000000            0.298354    164.083333
max       69.000000            0.511085    216.582051

Here is the plot:

I avoid calculating the player count, if data is missing from at least one server, which happens when a server is empty, but also when there are no births or deaths for a few minutes. So I just excluded these records, because it was the fastest solution. Hence the gaps. That could be improved, but I don't think it would change the results much.

I also tested it only with server 1 and get pretty much the same results:

         fertile_moms  ratio_fertile_moms  player_count
count  21601.000000        20966.000000  20966.000000
mean      14.521920            0.270206     54.213644
std        3.662977            0.058778      7.806979
min        0.000000            0.000000      1.000000
25%       12.000000            0.232278     50.285714
50%       15.000000            0.266667     57.857143
75%       17.000000            0.305085     59.800000
max       33.000000            0.831683     66.500000

Let me know, if this is what you needed. I'll be offline until tomorrow though. If my hangover will allow it, I can maybe find time on Jan 1st to work some more on this.

thundersen · 2018-12-31 13:34:15

betame wrote:

If you already grabbed that data, cool.
CSV works as a standard
either PM me or a link (google drive or?) for everyone here

Here's a link to a zip file with two csv files, lineages.csv and characters.csv.

The zip file is 34MB, the characters csv file is 185 MB when deflated, so you might have to break it up in a text editor or whatever before being able to use it in Excel.

betame · 2019-01-03 03:34:48

Neat, tyvm that's above and beyond.

Interesting I think I'm seeing %birthers increase a tiny bit during infertile hours, representing increased value in mothers competing for babies.

An average birther ratio of 27.5% doesn't change the relative math much; and the exact numbers for each situation could be gathered directly from life-log data.

In summary, the problems reducing birth rate related to this thread can be ranked by magnitude as follows:
Servers bouncing on/off > /dieing to become eve > nocturnal infertility

Saw this post. Jason's server distribution seems to be at odds with his vision.

jasonrohrer wrote:

[...]Every family line is competing for babies. This is a fundamental tenet of the game. [...]
But in order to properly compete, they need a fair chance to be interviewed. By the baby. And baby-keeping strategies need to be more viable. So baby-keeping culture, whatever that is, can develop through inter-cultural competition.

jasonrohrer · 2019-01-03 18:44:00

Thanks for going so deep here, everyone.

The double-bounce shown on server 4 around midnight on December 15 here is troublesome:

https://plot.ly/~thundersen/0.embed

Server3 has a pop cap of 160, so it's 10% threshold is 16, and its 50% threshold is 80.

You can see that Server3 got above 80.

Server2 got above 80 as well (look at the gold spike earlier), which is why server3 was brought into the mix at all. Of course, in the case of server2 and server3, everything was fine, because the population continued to rise. Server2 hits 80 at 3pm, then the rise continues, and server3 hits 80 at 10pm.

The problem is that there's a noisy plateau after that.

We finally get to the point where server3 hits 17, and at that point, server4 is at 36.

Server3's cut-off where it stops sending people on to server4 is 16 (10%), so we can assume it got there and that there's a gap in the data.

Now, a question about why the heck server4 had 36, when server3 only had 17. My first guess is that it has something to do with the way spreading works.

Server1 sends 50% of incoming players on to server2, which sends 50% of the remainder on to server3, which sends 50% of the remainder on to server4.

From this math, it seems like server4 and server3, especially, should have exactly the same expected population.

Except that there's one more factor here, and it's a big one:

Our random number generator is seeded by a given player's email address

This means that if you join the game under the same spreading conditions, you will always be sent to the same server, based on your email. You'll skip server1 (first coin flip always tails for you), skip server2 (second coin flip always tails for you), and then land on server3 (third coin flip always heads for you).

I'm not sure why I decided to do it this way. It just seemed like a sensible thing to do. If you play a few lives in a row, they will be on the same server. You could potentially make a pilgrimage back to your last village (or follow the bell) in the next life.

And there's an added benefit here, in that for twins, we can easily use the same logic to seed based on the twin code, so that we guarantee the twins are sent to the same server.

However, this means that if you're screwed into playing on server3, you're really screwed. Life after life there, for a few hours.

AND, I suspect that the population drop on 3 that happens at 10pm has "inertia" in the minds of the players who are still stuck playing there. Birth rate drops dramatically, villages die out, and a bunch of the "server3 assignees" simply stop playing there.

Though this is all just speculation. There may be something else going on.

See after the yellow server2 spike at 3pm, how server2 drops, but then gets "in step" with server3, and they rise together? That is what a 50/50 population split should look like.

I'm also not sure why server1 looks so hard-capped at 60. During the yellow spike at 3pm, why did server1 not rise with it at all? It should have been sharing 50% of new players with server2, so they should have risen together.

Here's a good example of what I'm talking about here:

As the pop falls up until 12:00, you can see an expected distribution, based on the spreading.

For example, at one point in the graph, there are 38, 23, and 12 players on server 1, 2, and 3, for a total of 75. Roughly 50% are on server1, 25% are on server2, and 12.5% are on server3. Roughly. Each subsequent server gets 50% of the remaining players.

But after Noon, server3 drops below 16 players, so it's taken out of rotation. Then, as expected, the pops on server1 and server2 rise together, sharing 50/50 of the incoming players. They are neck and neck.

Then, at 17:00, server1 crosses the 60 player mark (its 50%), at which point it stops rising, while server2 continues to rise up to 80, and then server3 is brought back into the mix again.

But WHY doesn't sever1 rise up to 80 along with server2 between 17:00 and 18:00?

jasonrohrer · 2019-01-03 18:54:03

An alternative algorithm here would be this:

When faced with a player joining the game, check the current total population, and figure out how many servers need to be used to balance that current load. If it's 150 players, say we decide on three servers.

Seed the random generator with that player's email, and send them to one of the top three servers in the list.

Later on, if the population drops to 120, we will realize that we only need two servers instead of three. Then everyone would be sent to server1 and server2, and server3 would die out.

A problem occurs right at the boundary between "three servers needed" and "two servers needed," where server3 will bounce between death and rebirth.

So thresholds would be needed. After deciding to start using three servers, we keep using them until we drop way below that threshold.

The other thing, and part of the reason I designed the reflector the way that I did, is that no global knowledge is needed to decide where to send a player. We can talk to each server on the list in turn, and make our decision about the player as soon as we have enough information to make it, without talking to all 15 servers every time.

We check server1 first. What's its current pop? Is it flagged as being in "spreading" mode or not? Then we send the player to server1, and if not, go on and consider server2.

jasonrohrer · 2019-01-03 19:48:54

Oh, I think I figured out why the population dips on server3 after 10pm in this graph, while server4's pop rises:

https://plot.ly/~thundersen/0.embed

Server3 simply has an older population at that point, because server4's population is brand new.

If we're splitting incoming babies 50/50, but your death rate is higher (because your population is older), then my population will rise, while yours will drop.

jasonrohrer · 2019-01-03 19:49:32

Also, I figured out why sever1's pop stops rising when it hits 60:

                if( mt_rand( 0, 1 ) == 0 ) {
                    return false;
                    }
                if( $twin_code == "" &&
                    $current / $max > $startSpreadingFraction ) {
                    // if not twins, send additional people to
                    // other servers until this one dies down below
                    // threshold

                    // so, what happens is this:
                    // once we pass threshold, all non-twins are sent to
                    // other servers
                    // When we die down below threshold, 50% of players
                    // are sent to other servers, until we get back
                    // below the stopSpreading fraction, and then we stop.
                    return false;
                    }

So that's working as intended.

CrazyEddie · 2019-01-03 20:09:16

Here's a design that might work:

On the reflector, keep state for "number of active servers" and "last reported population of server[x]". When a new connection arrives, add up the last reported population for all the active servers and divide by the number of active servers; if it's above some high threshold increase the number of active servers, and if it's below some low threshold decrease the number of active servers.

Then send the player to a random active server (seeded by email / twin code). Query that server for its current population and update the last-reported-population state for that server.

This will (roughly) evenly distribute the players across all servers while ensuring that there's (roughly) no more than X and no less than Y players per server. And it doesn't require querying every server on every connection - just one.

I would make the lower threshold very low, so that even when population is falling most of the servers still remain active. Deactivating a server sterilizes all the lineages on that server and is very depressing for anyone who happens to be on it at the time.

If the global population and active server count is close to an inflection point (either approaching or just past), send twins to Server1 instead of randomly, since there's a chance that the players might have connected at moments when there were a different number of active servers, which would cause the random seeding to send them to different servers.

CrazyEddie · 2019-01-03 20:20:45

jasonrohrer wrote:

Also, I figured out why sever1's pop stops rising when it hits 60:
[..]
So that's working as intended.

I think the code's stated intention differs from your intention as described in your July post.

In your post, the thresholds worked like this:
at 10% stop spreading,
at 50% start spreading via coinflip,
at >95% send it all to the next server without coinflipping.

But in the code I think you collapsed the last two thresholds into one, so that it became:
at 10% stop spreading,
at 50% start spreading via coinflip,
at >50% send it all to the next server without coinflipping.

jasonrohrer · 2019-01-03 20:31:23

The inflection point thing is possible in the current code, sending unlucky twins to separate servers. I'm assuming that, in that rare case, they notice that they're not finding each other, and try again. That's okay, for now.

Here's the question:

Low population (overall) means a low birth rate. Well, not exactly. It means fewer players coming in. Since fewer players coming in leads to a population drop, then the overall birth rate (per active player) should remain constant, except during times with the rate of players coming in is changing. As the rate rises, the birth rate goes up, temporarily. As the rate falls, the birth rate goes down, temporarily.

Do we really want the same thin population everywhere? If we pump up to 4 servers when the population goes above 200, do we really want to go all the way down to 20 players per server when the population drops back down to 80?

If there are still multiple lineages going per server, that means that every village will be a near ghost town.

If, on the other hand, we start turning off servers when the population drops, we artificially boost the birth rate on the remaining servers.

Furthermore, we allow for potentially larger villages to remain on the larger servers.

And finally, here's one more thing:

The general idea in this game is that villages/cultures COMPETE for incoming babies. A village with more warm/fed/fertile women will get more babies. If there are two villages, and one is "better" than the other in these terms, it will grow, and the other one will die out.

However, this does NOT happen across servers.

And this is another reason why we want the population concentrated on the fewest possible servers, when we can.

Otherwise, if we're spread out thin on four servers during low-pop times, the villages won't be able to compete for babies very well. Perhaps we'd eventually just end up with one village per server.

And furthermore, given the way the reflector is seeded, this would mean you'd always be born in the same village (or forced to be an Eve if you have a lineage ban).

Yes, we could ditch that seeding thing, and only use it for twins, but it's also useful when players disconnect (they will generally be reconnected to the same server).

In general, I see the huge problem here: Even if you are running your village perfectly, it can die for reasons outside of your control. And worse, given your luck of the draw with email-based seeding, you might find yourself ALWAYS playing in this situation (if you play at the same time of day, everyday, you might always get stuck on server4 right before a downturn).

jasonrohrer · 2019-01-03 20:48:20

CrazyEddie wrote:

I think the code's stated intention differs from your intention as described in your July post.

Absolutely.

At the time of the change, I was up against the reality that some servers were getting really slow (laggy) under high population load, and I needed a more direct and exact way of setting a hard limit.

The old code created a very fuzzy limit.

But yes, the result is that the server that just hit the limit for the first time gets no babies for quite a long while. This is not great.

jasonrohrer · 2019-01-03 21:38:04

Okay, I'm working on CrazyEddie's suggestion.

Will make it live in a bit, and we'll just have to see how it feels.

CrazyEddie · 2019-01-03 22:17:15

Regarding competition:

You can create inter-server competition by not seeding the server selection. With seeding, assuming a constant global population, a server's birth rate will be equal to its death rate; players who die on ServerX will be reborn on ServerX. But without seeding, all servers will have the same birth rate; each will get an even share of the global death rate. But each server will still have its own death rate. Servers that do a better job of keeping their population alive will see their population rise, and the others will fall. The same will be true of families within an individual server, with the added skewness of the yum and heat mechanisms.

Given that your birth parentage (who your mother is and where she is located) within a server is supposed to be random and independent, i.e. it's not expected that you'll keep getting reborn to the same family if there are multiple families on the server, I don't think there's any reason to make births "sticky" across servers.

The reconnect functionality is very good; perhaps there's a way it could be preserved during a life without creating sticky sessions between lives. Maybe the reflector could query the "sticky" server and find out if the player is alive? If so reconnect, if not re-randomize for a new connection.

Regarding thin populations:

How a town's population feels has a lot less to do with its population and much more to do with its birth rate. Basically, towns are either calm, booming, or dying. Absolute size means very little: a four-person Eve settlement can feel "full", a twelve-person end-stage town can feel "empty". But everyone can tell when the babies are coming fast and furious, and everyone gets depressed and miserable when the baby stream has dried up.

With the current reflector, every time a server is activated, that server has a boom and its upstream neighbor has a bust. And every time a server is de-activated, that server has a bust (and is exterminated) and its upstream neighbor has a boom. This is not a good dynamic.

If you spread the connections evenly across all the active servers, and allow them all to evenly share the global population increases and declines, then each one will feel roughly stable rather than having booms or busts. This is what Server1 and Server2 feel like almost all the time now, since both of them are usually held constant at their soft cap of 50% and therefore have neither booms nor busts.

And regarding competition, again:

There will always be at least two or three lineages on a server thanks to lineage bans (if the lifespans are long) or birth cooldowns (if the lifespans are short). So there will be at least two or three camps / settlements / towns competing for the incoming baby streams even within a single server, no matter how low the population gets.

jasonrohrer · 2019-01-03 23:32:30

Regarding the stickiness of given accounts on given servers...

I still want to mostly maintain the illusion that you are playing on "one big world."

There will hopefully be road networks and other things between towns in the future (I'm still working on making this more likely), so being born again into the same world is important.

And, a bit further down the line, there will be higher-tech transport that makes travel between villages feasible. In that case, you really WILL want to be born into the same world every time.

So, I'm not going to mess with that just yet.

jasonrohrer · 2019-01-03 23:38:56

Okay, this change is live.

It tracks a current number of "active" servers.

It computes the total population on those servers, and the total capacity on those servers.

If the total pop >= 50% total cap, it adds another server.

If the total pop <= 10% total cap, it makes the last server inactive.

It sends players to the servers evenly, weighted by each server's fraction of the total active server capacity. So, for example, server1 currently has a smaller capacity than the others, so it will get a smaller fraction of incoming players.

Email hashing to seed the RNG is still in place, so given the same conditions, you will always be sent to the same server.

Grim_Arbiter · 2019-01-03 23:51:30

jasonrohrer wrote:

There will hopefully be road networks and other things between towns in the future

There were two bell towns last month 2k tiles from each other that almost joined up by road. We were really close.

CrazyEddie · 2019-01-04 02:01:56

I love that you can make these kinds of changes more-or-less at the drop of a hat.

I hope this one works out! It'll be interesting to see what kind of effect it has on things, although honestly it may be difficult to really know.

jasonrohrer · 2019-01-04 02:31:24

Well, we can look at the graphs, right?

CrazyEddie · 2019-01-04 02:43:46

Well, that'll tell you whether each server's population is growing and shrinking as intended. But what effect that has on the common impression that "My family keeps dying because the babies dry up" will be harder to gauge, since that's essentially anecdotal at this point.

The necessary data is in the birth and death logs, but I don't think anyone has yet done an analysis that could answer those sorts of questions objectively and definitively. thundersen is doing amazing work with that data (see his visualization of the birth locations) so it'll probably get teased out before long.

betame · 2019-01-04 02:54:36

The graphs won't lie, and I expect this change to help. Thanks for lending attention!

But there is one daily bouncing issue I predict, which can be fixed by making server 1 size be the largest or by increasing the 50% cap.

We should expect a daily cycle's maxiumum to be ~2.5x its minimum, looking at thundersen's graphs. [It might become smaller if the game spreads more into other time zones]
Therefore,
When the total player population is ever in a range where its minimum is ~28, it will daily trigger server 2 to be removed, and before it hits its daily maximum of ~70 it will trigger server 2 to be added (and doomed).
---Note this is the only range it would be a problem.

If server 2 were smaller than server 1, then this will no longer be a problem,
or if the 50% cap were increased, this should no longer be a problem.

thundersen · 2019-01-05 11:49:57

Yay! Glad this got fixed. Thanks Jason and CrazyEddie!

First impression is good:

This graph here is still being updated at least once per day.

Any ideas why server 4 didn't come back up after the 2nd update? I think that is a good thing. The fewer servers the better for that one world experience. I think it would still be worth considering switching to fewer more powerful servers, so that one or two servers could hold the entire population. If there were only two active servers, all you'd have to do besides spreading births 50/50 would be to keep twins together. So this should also simplify the reflector and make it easier to reason about.

But hey, the current system works and it seems fine with the fix. Just mentioning this alternative as a possible future improvement.

CrazyEddie wrote:

But what effect that has on the common impression that "My family keeps dying because the babies dry up" will be harder to gauge, since that's essentially anecdotal at this point.
The necessary data is in the birth and death logs[...]

I agree. What would be a good metric to track this though? Would something like % FERTILE LINEAGES be enough? I could do that. Consider a lineage fertile as long as there's at least one female who hasn't reached end of fertility yet. That doesn't tell us anything about the reason for the lineage end, but we could verify, if there were fewer sudden drops in that metric over the day, as intended by this update, right?

thundersen · 2019-01-06 15:15:17

Any ideas why server 4 got starved yesterday starting at about 14:30? Looks suspicious to me.

Among others that incident ended the unfortunate Foley family (not the one that LydiaVDH posted about, but a second one started by the same player): http://lineage.onehouronelife.com/serve … id=2695216

SSDarkMoon · 2019-03-05 19:05:15

hi,why there's a huge drop after jan26?
https://plot.ly/~thundersen/4.embed

One Hour One Life Forums

#26 2018-12-30 02:31:58

Re: Server population over time

#27 2018-12-30 12:33:22

Re: Server population over time

#28 2018-12-30 15:28:48

Re: Server population over time

#29 2018-12-31 13:34:15

Re: Server population over time

#30 2019-01-03 03:34:48

Re: Server population over time

#31 2019-01-03 18:44:00

Re: Server population over time

#32 2019-01-03 18:54:03

Re: Server population over time

#33 2019-01-03 19:48:54

Re: Server population over time

#34 2019-01-03 19:49:32

Re: Server population over time

#35 2019-01-03 20:09:16

Re: Server population over time

#36 2019-01-03 20:20:45

Re: Server population over time

#37 2019-01-03 20:31:23

Re: Server population over time

#38 2019-01-03 20:48:20

Re: Server population over time

#39 2019-01-03 21:38:04

Re: Server population over time

#40 2019-01-03 22:17:15

Re: Server population over time

#41 2019-01-03 23:32:30

Re: Server population over time

#42 2019-01-03 23:38:56

Re: Server population over time

#43 2019-01-03 23:51:30

Re: Server population over time

#44 2019-01-04 02:01:56

Re: Server population over time

#45 2019-01-04 02:31:24

Re: Server population over time

#46 2019-01-04 02:43:46

Re: Server population over time

#47 2019-01-04 02:54:36

Re: Server population over time

#48 2019-01-05 11:49:57

Re: Server population over time

#49 2019-01-06 15:15:17

Re: Server population over time

#50 2019-03-05 19:05:15

Re: Server population over time

Board footer