Public access to all life log data

jasonrohrer · 2018-07-04 06:21:09

Here you go, folks:

http://publicdata.onehouronelife.com/

This data can be used to analyze all kinds of things. Life expectancy, birth rate, player population size, population densities on the map, average number of games per player, average number of games before first old-age death, etc. etc. etc.

You can effectively construct every family tree in the game from this information.

I've replaced all email addresses with secret-salted SHA1 hashes. Thus, even if you know an email address (like your own), you cannot look yourself up. But it does connect all actions of a given player together via a unique identifier. The secret is held on the server and not in the public github. If there is a breach that accessed this secret salt, hash-reversing could occur for known email address, but in a breach, they'd probably have access to the database anyway, where emails are stored in the clear.

Okay, data format:

D 1530664297 914919 9e0b3888a1eab461062d25fc7da846e17531df25 age=12.05 F (238839,15823) hunger pop=64

B 1530664299 915002 0c3152a280f2bed64c71482062924702df9f41a5 F (250890,40288) parent=914761 pop=66 chain=12

B 1530664300 915003 9e0b3888a1eab461062d25fc7da846e17531df25 F (251640,16496) noParent pop=67 chain=1

D 1530664301 914897 e11fd4f020397a322bca64093713e969d85b140c age=15.05 F (257741,70628) killer_914925 pop=66

B are birth lines with these data elements:
Unix time
Player ID
email hash
Female/male
Birth coords
Parent player ID (or noParent)
population at time of birth
Chain-to-Eve length

D are death lines with these data elements:
Unix time
Player ID
email hash
Age at death in years
Female/male
Death coords coords
cause of death (including player ID of killer, if applicable)
population at time of death

Player IDs are unique per life, and increment for each player spawn, but NOT unique across servers. Each server is running its own ID sequence starting at 0.

Note that there have been some problems on Server1 that have caused Player IDs to go back to 0 occasionally, leading to some repeated player IDs over time (Server1 ran out of disk space a few times and couldn't save the next player ID to disk).

UNIX times are GMT.

And this is updated nightly with the latest data via a cron job at 8:30 GMT.

jasonrohrer · 2018-07-04 17:01:06

Nothing here?

YAHG, isn't this what you were dreaming about?

TheRedBug · 2018-07-04 17:17:31

He's comming. I hear his footsteps.

Anshin · 2018-07-04 17:17:35

How do I use this?
How would I look myself up?
Can I find out totals about myself, like how many hours I've played?
???

TheRedBug · 2018-07-04 17:22:22

You can see how many hours you have played. If you have made a review of the game, and go to the site frontpage, it'll tell you

IAmBlue · 2018-07-04 17:26:47

IAMBLUE

jasonrohrer · 2018-07-04 17:31:42

If you want to look up yourself, you can sha1 hash your own email address using a tool like this:

http://www.sha1-online.com/

After that, you can search for that hash in these logs to find your births and deaths.

NOTE: this no longer works, because the email hashes now use a secret salt.

ryanb · 2018-07-04 18:16:44

This is awesome, thanks Jason! I'm tempted to make a player stats page with information like: average life length, cause of death percents, average length of descendants, etc.

jasonrohrer · 2018-07-04 18:48:26

I've realized that posting hashed email addresses is probably not a great idea:

https://freedom-to-tinker.com/2018/04/0 … addresses/

So I'm going to take the data down now and re-work the format to use something else as a unique, per-user identifier.

jasonrohrer · 2018-07-04 19:35:44

Okay, fixed. Email hashes now use a secret salt. So even for a known email address, you can't determine whether someone owns the game or their play patterns from this list.

YAHG · 2018-07-05 04:01:57

jasonrohrer wrote:

YAHG, isn't this what you were dreaming about?

Lol I said I thought YOU might like to see those graphs...

You just put in you know how many hours into the tutorial and then posted the most recent data I assumed because you wanted to know if it was working as intended.

I assume the goal of the tutorial is that people would be more competent players. Assuming competent people live longer etc. those graphs would
show the results if there were any.

Also I had to work all day.. Then 2 hours of traffic on the way home and on my home street I ran over an unexploded firework.. BOOM

Luckily no damage and now I can finally drink my beer..

sc0rp · 2018-07-05 12:47:07

jasonrohrer wrote:

Okay, fixed. Email hashes now use a secret salt. So even for a known email address, you can't determine whether someone owns the game or their play patterns from this list.

If it's single salt you can use rainbow tables to try to get around it. Safe way is to have separate salt per user.

[Just paranoid view - I know nobody will spend time on it.]

sc0rp · 2018-07-05 12:48:21

jasonrohrer wrote:

Nothing here?
YAHG, isn't this what you were dreaming about?

Just give us some time Jason. We need to parse this stuff into something usefull, etc. and we are not working full time on OHOL

Chard · 2018-07-05 15:00:11

jasonrohrer wrote:

Here you go, folks:
http://onehouronelife.com/publicLifeLogData/

Some files appear truncated: e.g. http://onehouronelife.com/publicLifeLog … turday.txt

jasonrohrer · 2018-07-05 15:16:47

Since the salt is secret (only I have access to it, unless there's a breach), then rainbow tables will not work. I mean, you'd have to rainbow table the salt too, but I made it very long.

Actually, salt is the wrong word for this. What I'm doing is a secret pepper.

And if there's a breach, the email addresses are sitting in the database anyway. The point of the peppered hash is to eliminate email exposure in published data.

Chard:

Data is cut off the same way in the original file. Not sure what happened there. Maybe the server crashed? Anyway, it's the best info I have about May 19.

Chard · 2018-07-05 15:22:32

Other weird lines to check out:

server1, June 8th, line 7123
server1, March 31st, line 5364
server4, April 8th, line 135

jasonrohrer · 2018-07-05 15:40:26

March 31 is very weird. christopherB, a non-email address, suddenly appeared in the place of an email that started "christopherb...." This somehow happened mid-life, server-side. I have no idea how this is possible. Anyway, my email-replacement script let it slip through, because it was not a real email address.

June 8 looks like another crash.

April 8 is caused by someone having non-ascii characters in their email address. Which is weird, because they don't appear that way in the account database. But they are logged several times that day with the non-ascii character at the start of their email. And no one else is ever logged with that particular character at the start of their email (character is « not sure if that will show up here).

Not sure how they could type this into the client. Possible that some kind of client-side corruption is the cause. Unlikely that it is server-side corruption, because it shows up for multiple births (which would all be using different memory locations).

ANYWAY, this data is likely not 100% clean.

Processing scripts probably need to go through it line-by-line and ignore any lines that don't match the format.

Chard · 2018-07-05 15:46:18

jasonrohrer wrote:

March 31 is very weird. christopherB, a non-email address, suddenly appeared in the place of an email that started "christopherb...." This somehow happened mid-life, server-side. I have no idea how this is possible. Anyway, my email-replacement script let it slip through, because it was not a real email address.

Probably an email that started "christopher" followed by a crash then the first line after restart is "B ...". I've seen several instances of that. Server should probably put a newline into the lifelog when it opens it in case it wasn't closed cleanly.

jasonrohrer wrote:

June 8 looks like another crash.
April 8 is caused by someone having non-ascii characters in their email address. Which is weird, because they don't appear that way in the account database. But they are logged several times that day with the non-ascii character at the start of their email. And no one else is ever logged with that particular character at the start of their email (character is « not sure if that will show up here).

Also shows up on server2 April 8th. My full list of errors:

** /home/richard/lifelogs/publicLifeLogData/lifeLog_server3.onehouronelife.com/2018_03March_09_Friday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server3.onehouronelife.com/2018_03March_08_Thursday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server3.onehouronelife.com/2018_03March_10_Saturday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_05May_19_Saturday.txt:10060 -- D 1526743767 572797
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_06June_08_Friday.txt:7122 -- B 1528476297 754649 9e4a4d35bd0718171a97a6d847bc3c0d9255aa4d F (224076,7
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:5363 -- D 1522510172 161342 christopherB 1522511234 161377 dd3569a0626f77ac87e0359a2bae6a79de506685 M (-224,-584) parent=161310 pop=39 chain=3
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:5406 -- B 1522511790 161B 1522512131 161407 e9b726020d59480113d793f1be2365803f1d7e6d F (97,-30) noParent pop=1 chain=1
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:9223 -- B 1522525197 163352 3a1107ad0ac9b22c00f38a3b4ed89c9e49bca5cf 1522525450 163386 67dac6856666ae544a4799555eb5a98c10ec2784 age=1.14 F (122,191) hunger pop=38
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:9588 -- B 1522526499 163602 02d1829b7179139f580b5b529c035c001f6141d5 F (129,-187) parent=163478 1522526775 163636 6a071b942f2656254014cd38ea0346d5cc65bf60 F (97,-30) noParent pop=1 chain=1
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:9678 -- B 1522526972B 1522527598 163800 c028138974518a40bdbb6daaae4913d528d1b88a F (-398,-90) noParent pop=40 chain=1
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_31_Saturday.txt:9723 -- B 1B 1522531431 163826 c5f37b2a7b57e9db18302729e9869477698de488 F (149,-113) noParent pop=1 chain=1
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_07July_04_Wednesday.txt contains byte 255, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_03March_10_Saturday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server1.onehouronelife.com/2018_06June_02_Saturday.txt contains byte 246, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:134 -- B 1523148461 121552 c5a390dc911d460a846e7c939148d0c633265aa0 F (-21012,-27988) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=37 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:168 -- B 1523148585 121568 1f0a0b5efef6aef501f74073a07837711b81107c M (-21010,-27993) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=35 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:198 -- B 1523148767 121584 5eee093fcac6b1d2bdad8f14c10fb75c51467b15 M (-20999,-28013) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=39 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:237 -- B 1523149032 121604 0c4c33794d909c49a8a94fb234a0463f02aec46b M (-21033,-27995) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=40 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:258 -- B 1523149130 121614 7b8755ced056f153bfe7a6dfbf43568d70c5279b F (-21006,-27997) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=38 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:278 -- B 1523149295 121625 ccbd477f5974c2c8ef2c2a2e51a391f3542e21b2 F (-20998,-27992) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=39 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server4.onehouronelife.com/2018_04April_08_Sunday.txt:296 -- B 1523149370 121633 f1606b8e2aed395c8b0cf821050da934e8ee6fff F (-20992,-27999) 770a8f9ffde37e59afd6aef97f0bdf25f6561642 pop=38 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server5.onehouronelife.com/2018_03March_09_Friday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server5.onehouronelife.com/2018_03March_10_Saturday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_03March_11_Sunday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_07July_05_Thursday.txt contains byte 255, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_03March_09_Friday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_03March_08_Thursday.txt contains byte 167, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_04April_08_Sunday.txt:119 -- B 1523148113 193675 c33ed74fc73efa1d276e71eb065d87a56d45f4ca F (9357,5480) f97686ead37801376cb99e1b4a50bc8b7601cec8 pop=25 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_04April_08_Sunday.txt:172 -- B 1523148321 193703 6fc2435c36d7188eec6c9e93ba91e18c8f083c48 M (8820,5391) 67f6a49a752df4efbb9d98046447725411af4e32 pop=30 chain=2
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_03March_12_Monday.txt:19478 -- D 1520888253 87113 cfbdba9cc60ea334e151a221d55784cf1875f6d3 age=6.9
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_07July_04_Wednesday.txt contains byte 255, ignoring non-ascii characters
** /home/richard/lifelogs/publicLifeLogData/lifeLog_server2.onehouronelife.com/2018_03March_14_Wednesday.txt contains byte 167, ignoring non-ascii characters

jasonrohrer wrote:

ANYWAY, this data is likely not 100% clean.
Processing scripts probably need to go through it line-by-line and ignore any lines that don't match the format.

Understood.

jasonrohrer · 2018-07-05 16:07:28

Good catch on the christopherB thing... not sure why I didn't think of that!

Chard · 2018-07-05 16:27:58

Player IDs are non-unique on servers 1, 5, 6, 7, 8, 10, 12, 13 and 14.

Basically can't rely on player ID to match birth and death lines. There are other routes though.

jasonrohrer · 2018-07-05 16:42:18

Really? Not unique on server 12?

I know that server1 ran out of disk space, and I had to manually re-set the next player ID back up after it went back to 0 by accident.

jasonrohrer · 2018-07-05 16:48:41

Yeah, I see the IDs restart on April 16 in server 12.... why? Gotta figure that out.

Chard · 2018-07-05 16:57:20

Yeah, lots of IDs repeating.

I've also go death lines with no corresponding birth line anywhere.

server3, id=12315 is an example.

TrustyWay · 2018-07-05 17:00:43

Is there an id that shows a killer who keep killing everybody all the time ? We should ban people totally crazy who only does that.

Only one ban in whole OHOL history, for the good of everyone. Please Jason, I know you don't want to do that but only one after all these months of stress.

Is there anyone who knows how to search for a mass mass murder in this mess ?

Last edited by TrustyWay (2018-07-05 17:05:41)

jasonrohrer · 2018-07-05 17:07:12

So.... April 16 was the day that a bunch of servers were rebooted by Linode without warning.

The server is supposed to save the next ID to disk constantly as it is updated, but during a hard reboot, I guess this didn't happen for some reason. Maybe the reboot happened right at the moment it was trying to write the file.

Weird that it happened the same way for a bunch of servers on the same day, but there you have it.

Fortunately, when calculating most things, it doesn't matter, because the ID reset jumps so far backward, and people never live more than an hour... i.e., IDs are safe to use for local parent matching within the same day or two, always.

12315 on server 3 is born the previous day, probably around midnight, on March 5.

The logs roll over each day, and sometimes lives cross the roll-over point.

One Hour One Life Forums

#1 2018-07-04 06:21:09

Public access to all life log data

#2 2018-07-04 17:01:06

Re: Public access to all life log data

#3 2018-07-04 17:17:31

Re: Public access to all life log data

#4 2018-07-04 17:17:35

Re: Public access to all life log data

#5 2018-07-04 17:22:22

Re: Public access to all life log data

#6 2018-07-04 17:26:47

Re: Public access to all life log data

#7 2018-07-04 17:31:42

Re: Public access to all life log data

#8 2018-07-04 18:16:44

Re: Public access to all life log data

#9 2018-07-04 18:48:26

Re: Public access to all life log data

#10 2018-07-04 19:35:44

Re: Public access to all life log data

#11 2018-07-05 04:01:57

Re: Public access to all life log data

#12 2018-07-05 12:47:07

Re: Public access to all life log data

#13 2018-07-05 12:48:21

Re: Public access to all life log data

#14 2018-07-05 15:00:11

Re: Public access to all life log data

#15 2018-07-05 15:16:47

Re: Public access to all life log data

#16 2018-07-05 15:22:32

Re: Public access to all life log data

#17 2018-07-05 15:40:26

Re: Public access to all life log data

#18 2018-07-05 15:46:18

Re: Public access to all life log data

#19 2018-07-05 16:07:28

Re: Public access to all life log data

#20 2018-07-05 16:27:58

Re: Public access to all life log data

#21 2018-07-05 16:42:18

Re: Public access to all life log data

#22 2018-07-05 16:48:41

Re: Public access to all life log data

#23 2018-07-05 16:57:20

Re: Public access to all life log data

#24 2018-07-05 17:00:43

Re: Public access to all life log data

#25 2018-07-05 17:07:12

Re: Public access to all life log data

Board footer